Task runners are the heroes (or villains, depending on your point of
view) that quietly toil behind most web and mobile applications. Task
runners provide value through the automation of numerous development
tasks such as concatenating files, spinning up development servers and
compiling code. In this article, we’ll cover Grunt, Gulp, Webpack and
npm scripts. We’ll also provide some examples of each one to get you
started. Near the end, I’ll throw out some easy wins and tips for
integrating ideas from this post into your application.
There is a sentiment that task runners, and JavaScript advances in
general, are over-complicating the front-end landscape. I agree that
spending the entire day tweaking build scripts isn’t always the best use
of your time, but task runners have some benefits when used properly
and in moderation. That’s our goal in this article, to quickly cover the
basics of the most popular task runners and to provide solid examples
to kickstart your imagination regarding how these tools can fit in your
workflow.
A Note on the Command Line
Task runners and build tools are primarily command-line tools.
Throughout this article, I’ll assume a decent level of experience and
competence in working with the command line. If you understand how to
use common commands like
cd,
ls,
cp and
mv,
then you should be all right as we go through the various examples.Let’s kick things off with the granddaddy of them all: Grunt.
Grunt
Grunt was the first popular
JavaScript-based task runner. I’ve been using Grunt in some form since
2012. The basic idea behind Grunt is that you use a special JavaScript
file,
Gruntfile.js, to configure various plugins to
accomplish tasks. It has a vast ecosystem of plugins and is a very
mature and stable tool. Grunt has a
fantastic web directory
that indexes the majority of plugins (about 5,500 currently). The
simple genius of Grunt is its combination of JavaScript and the idea of a
common configuration file (like a makefile), which has allowed many
more developers to contribute to and use Grunt in their projects. It
also means that Grunt can be placed under the same version control
system as the rest of the project.
Grunt is battle-tested and stable. Around the time of writing,
version 1.0.0 was released,
which is a huge accomplishment for the Grunt team. Because Grunt
largely configures various plugins to work together, it can get tangled
(i.e. messy and confusing to modify) pretty quickly. However, with a
little care and organization (breaking tasks into logical files!), you
can get it to do wonders for any project.
In the rare case that a plugin isn’t available to accomplish the task you need, Grunt provides
documentation on how to write your own plugin. All you need to know to create your own plugin is JavaScript and the
Grunt API. You’ll almost never have to create your own plugin, so let’s look at how to use Grunt with a pretty popular and useful plugin!
Let’s look at how Grunt actually works. Running
grunt in the command line will trigger the Grunt command-line program that looks for
Gruntfile.js in the root of the directory. The
Gruntfile.js contains the configuration that controls what Grunt will do. In this sense,
Gruntfile.js can be seen as a kind of cookbook that the cook (i.e. Grunt, the program) follows; and, like any good cookbook,
Gruntfile.js will contain many recipes (i.e. tasks).
We’re going to put Grunt through the paces by using the
Grunticon plugin to generate icons for a hypothetical web app. Grunticon takes in a directory of SVGs and spits out several assets:
- a CSS file with the SVGs base-64-encoded as background images;
- a CSS file with PNG versions of the SVGs base-64-encoded as background images;
- a CSS file that references an individual PNG file for each icon.
The three different files represent the various capabilities of
browsers and mobile devices. Modern devices will receive the
high-resolution SVGs as a single request (i.e. a single CSS file).
Browsers that don’t handle SVGs but handle base-64-encoded assets will
get the base-64 PNG style sheet. Finally, any browsers that can’t handle
those two scenarios will get the “traditional” style sheet that
references PNGs. All this from a single directory of SVGs!
The configuration of this task looks like this:
module.exports = function(grunt) {
grunt.config("grunticon", {
icons: {
files: [
{
expand: true,
cwd: 'grunticon/source',
src: ["*.svg", ".png"],
dest: 'dist/grunticon'
}
],
options: [
{
colors: {
"blue": "blue"
}
}
]
}
});
grunt.loadNpmTasks('grunt-grunticon');
};
Let’s walk through the various steps here:
- You must have Grunt installed globally.
- Create the
Gruntfile.js file in the root of the project. It’s best to also install Grunt as an npm dependency in your package.json file along with Grunticon via npm i grunt grunt-grunticon --save-dev.
- Create a directory of SVGs and a destination directory (where the built assets will go).
- Place a small script in the
head of your HTML, which will determine what icons to load.
Here is what your directory should look like before you run the Grunticon task:
|-- Gruntfile.js
|-- grunticon
| `-- source
| `-- logo.svg
`-- package.json
Once those things are installed and created, you can copy the code snippet above into
Gruntfile.js. You should then be able to run
grunt grunticon from the command line and see your task execute.
The snippet above does a few things:
- adds a new
config object to Grunt on line 32 named grunticon;
- fills out the various options and parameters for Grunticon in the
icons object;
- finally, pulls in the Grunticon plugin via
loadNPMTasks.
Here is what your directory should look like post-Grunticon:
|-- Gruntfile.js
|-- dist
| `-- grunticon
| |-- grunticon.loader.js
| |-- icons.data.png.css
| |-- icons.data.svg.css
| |-- icons.fallback.css
| |-- png
| | `-- logo.png
| `-- preview.html
|-- grunticon
| `-- source
| `-- logo.svg
`-- package.json
There you go — finished! In a few lines of configuration and a couple
of package installations, we’ve automated the generation of our icon
assets! Hopefully, this begins to illustrate the power of task runners:
reliability, efficiency and portability.
Gulp: LEGO Blocks For Your Build System
Gulp emerged sometime after Grunt and aspired to be a build tool that
wasn’t all configuration but actual code. The idea behind code over
configuration is that code is much more expressive and flexible than the
modification of endless config files. The hurdle with Gulp is that it
requires more technical knowledge than Grunt. You will need to be
familiar with the
Node.js streaming API and be comfortable writing basic JavaScript.
Gulp’s use of Node.js streams is the main reason it’s faster than
Grunt. Using streams means that, instead of using the file system as the
“database” for file transformations, Gulp uses in-memory
transformations. For more information on streams, check out the
Node.js streams API documentation, along with the
stream handbook.
As in the Grunt section, we’re going to put Gulp through the paces
with a straightforward example: concatenating our JavaScript modules
into a single app file.
Running Gulp is the same as running Grunt. The
gulp command-line program will look for the cookbook of recipes (i.e.
Gulpfile.js) in the directory in which it’s run.
Limiting the number of requests each page makes is considered a web
performance best practice (especially on mobile). Yet collaborating with
other developers is much easier if functionality is split into multiple
files. Enter task runners. We can use Gulp to combine the multiple
files of JavaScript for our application so that mobile clients have to
load a single file, instead of many.
Gulp has the same massive ecosystem of plugins as Grunt. So, to make this task easy, we’re going to lean on the
gulp-concat plugin. Let’s say our project’s structure looks like this:
|-- dist
| `-- app.js
|-- gulpfile.js
|-- package.json
`-- src
|-- bar.js
`-- foo.js
Two JavaScript files are in our
src directory, and we want to combine them into one file,
app.js, in our
dist/ directory. We can use the following Gulp task to accomplish this.
var gulp = require('gulp');
var concat = require('gulp-concat');
gulp.task('default', function() {
return gulp.src('./src/*.js')
.pipe(concat('app.js'))
.pipe(gulp.dest('./dist/'));
});
The important bits are in the
gulp.task callback. There, we use the
gulp.src API to get all of the files that end with
.js in our
src directory. The
gulp.src API returns a stream of those files, which we can then pass (via the
pipe API) to the gulp-concat plugin. The plugin then concatenates all of the files in the stream and passes it on to the
gulp.dest function. The
gulp-dest function simply writes the input it receives to disk.
You can see how Gulp uses streams to give us “building blocks” or
“chains” for our tasks. A typical Gulp workflow looks like this:
- Get all files of a certain type.
- Pass those files to a plugin (concat!), or do some transformation.
- Pass those transformed files to another block (in our case, the
dest block, which ends our chain).
As in the Grunt example, simply running
gulp from the root of our project directory will trigger the
default task defined in the
Gulpfile.js file. This task concatenates our files and let’s us get on with developing our app or website.
Webpack
The newest addition to the JavaScript task runner club is
Webpack.
Webpack bills itself as a “module bundler,” which means it can
dynamically build a bundle of JavaScript code from multiple separate
files using module patterns such as the
CommonJS pattern. Webpack also has plugins, which it calls
loaders.
Webpack is still fairly young and has rather dense and confusing documentation. Therefore, I’d recommend
Pete Hunt’s Webpack repository
as a great starting point before diving into the official
documentation. I also wouldn’t recommend Webpack if you are new to task
runners or don’t feel proficient in JavaScript. Those issues aside, it’s
still a more specific tool than the general broadness of Grunt and
Gulp. Many people use Webpack alongside Grunt or Gulp for this very
reason, letting Webpack excel at bundling modules and letting Grunt or
Gulp handle more generic tasks.
Webpack ultimately lets us write Node.js-style code for the browser, a
great win for productivity and making a clean separation of concerns in
our code via modules. Let’s use Webpack to achieve the same result as
we did with the Gulp example, combining multiple JavaScript files into
one app file.
Webpack is often used with
Babel to
transpile ES6 code to ES5. Transpiling code from ES6 to ES5 lets
developers use the emerging ES6 standard while serving up ES5 to
browsers or environments that don’t fully support ES6 yet. However, in
this example, we’ll focus on building a simple bundle of our two files
from the Gulp example. To begin, we need to install Webpack and create a
config file,
webpack.config.js. Here’s what our file looks like:
module.exports = {
entry: "./src/foo.js",
output: {
filename: "app.js",
path: "./dist"
}
};
In this example, we’re pointing Webpack to our
src/foo.js file to begin its work of walking our dependency graph. We’ve also updated our
foo.js file to look like this:
var bar = require("./bar");
var foo = function() {
console.log('foo');
bar();
};
module.exports = foo;
And we’ve updated our
bar.js file to look like this:
var bar = function() {
console.log('bar');
};
module.exports = bar;
This is a very basic CommonJS example. You’ll notice that these files
now “export” a function. Essentially, CommonJS and Webpack allow us to
begin organizing our code into self-contained modules that can be
imported and exported throughout our application. Webpack is smart
enough to follow the import and export keywords and to bundle everything
into one file,
dist/app.js. We no longer need to maintain a
concatenation task, and we simply need to adhere to a structure for our
code instead. Much better!
Extending
Webpack is akin to Gulp in that “It’s just JavaScript.” It can be
extended to do other task runner tasks via its loader system. For
instance, you can use
css-loader and
sass-loader to compile Sass into CSS and even to use the Sass in your JavaScript by
overloading the require CommonJS pattern!
However, I typically advocate for using Webpack solely to build
JavaScript modules and for using another more general-purpose approach
for task running (for example, Webpack and npm scripts or Webpack and
Gulp to handle everything else).
npm Scripts
npm scripts are the
latest hipster craze, and for good reason. As we’ve seen with all of
these tools, the number of dependencies they might introduce to a
project could eventually spin out of control. The first post I saw
advocating for npm scripts as the entry point for a build process was by
James Halliday.
His post perfectly sums up the ignored power of npm scripts (emphasis mine):
There are some fancy tools for doing build automation on JavaScript
projects that I’ve never felt the appeal of because the lesser-known npm run command has been perfectly adequate for everything I’ve needed to do while maintaining a very tiny configuration footprint.
Did you catch that last bit at the end? The primary appeal of npm
scripts is that they have a “very tiny configuration footprint.” This is
one of the main reasons why npm scripts have started to catch on
(almost four years later, sadly). With Grunt, Gulp and even Webpack, one
eventually begins to drown in plugins that wrap binaries and double the
number of dependencies in a project.
Keith Cirkel has the
go-to tutorial
on using npm to replace Grunt or Gulp. He provides the blueprint for
how to fully leverage the power of npm scripts, and he’s introduced an
essential plugin,
Parallel Shell (and a
host of others just like it).
An Example
In our section about Grunt, we took the popular module Grunticon and
created SVG icons (with PNG fallbacks) in a Grunt task. This used to be
the one pain point with npm scripts for me. For a while, I would keep
Grunt installed for projects just to use Grunticon. I would literally
“shell out” to Grunt in my npm task to achieve task-runner inception
(or, as we started calling it at work, a build-tool turducken).
Thankfully,
The Filament Group, the fantastic group behind Grunticon, released a standalone (i.e. Grunt-free) version of their tool,
Grunticon-Lib. So, let’s use it to create some icons with npm scripts!
This example is a little more advanced than a typical npm script
task. A typical npm script task is a call to a command-line tool, with
the appropriate flags or config file. Here’s a more typical task that
compiles our Sass to CSS:
"sass": "node-sass src/scss/ -o dist/css",
See how it’s just one line with various options? No task file needed, no build tool to spin up — just
npm run sass
from the command line, and you’re Sass is now CSS. One really nice
feature of npm scripts is how you can chain script tasks together. For
instance, say we want to run some task before our Sass task runs. We
would create a new script entry like this:
"presass": "echo 'before sass',
That’s right: npm understands the
pre- prefix. It also understands the
post- prefix. Any script entry with the same name as another script entry with a
pre- or
post- prefix will run before or after that entry.
Converting our icons will require an actual Node.js file. It’s not too serious, though. Just create a
tasks directory, and create a new file named
grunticon.js or
icons.js
or whatever makes sense to those working on the project. Once the file
is created, we can write some JavaScript to fire off our Grunticon
process.
Note: All of these examples use ES6, so we’re going to use babel-node
to run our task. You can easily use ES5 and Node.js, if that’s more
comfortable.
import icons from "grunticon-lib";
import globby from "globby";
let files = globby.sync('src/icons/*');
let options = {
colors: {
"blue": "blue"
}
};
let icon = new icons(files, 'dist/icons', options);
icon.process();
Let’s get into the code and figure out what’s going on.
- We
import (i.e. require) two libraries, grunticon-lib and globby. Globby is one of my favorite tools, and it makes working with files and globs so easy. Globby enhances Node.js Glob (select all JavaScript files via ./*.js) with Promise support. In this case, we’re using it to get all files in the src/icons directory.
- Once we do that, we set a few options in an
options object and then call Grunticon-Lib with three arguments:
- the icon files,
- the destination,
- the options.
The library takes over and chews away on those icons and eventually creates the SVGs and PNG versions in the directory we want.
- We’re almost done. Remember that this is in a separate file, and we
need to add a “hook” to call this file from our npm script, like this:
"icons": "babel-node tasks/icons.js".
- Now we can run
npm run icons, and our icons will get created every time.
npm scripts offer a similar level of power and flexibility as other task runners, without the plugin debt.
Breakdown Of Task Runners Covered Here
| Tool |
Pros |
| Grunt |
No real programming knowledge needed |
| Gulp |
Configure tasks with actual JavaScript and streams |
| |
|
| Webpack |
Best in class at module-bundling |
| npm scripts |
Direct interaction with command-line tools. |
Some Easy Wins
All of these examples and task runners might seem overwhelming, so
let’s break it down. First, I hope you don’t take away from this article
that whatever task runner or build system you are currently using needs
to be instantly replaced with one mentioned here. Replacing important
systems like this shouldn’t be done without much consideration. Here’s
my advice for upgrading an existing system: Do it incrementally.
Wrapper Scripts!
One incremental approach is to look at writing a few “wrapper” npm
scripts around your existing task runners to provide a common vocabulary
for build steps that is outside of the actual task runner used. A
wrapper script could be as simple as this:
{
"scripts": {
"start": "gulp"
}
}
Many projects utilize the
start and
test
npm script blocks to help new developers get acclimatized quickly. A
wrapper script does introduce another layer of abstraction to your task
runner build chain, yet I think it’s worth being able to standardize
around the npm primitives (e.g.
test). The npm commands have better longevitiy than an individual tool.
Sprinkle in a Little Webpack
If you or your team are feeling the pain of maintaining a brittle
“bundle order” for your JavaScript, or you’re looking to upgrade to ES6,
consider this an opportunity to introduce Webpack to your existing
task-running system. Webpack is great in that you can use as much or as
little of it as you want and yet still derive value from it. Start just
by having it bundle your application code, and then add babel-loader to
the mix. Webpack has such a depth of features that it’ll be able to
accommodate just about any additions or new features for quite some
time.
Easily Use PostCSS With npm Scripts
PostCSS is a
great collection of plugins that transform and enhance CSS once it’s
written and preprocessed. In other words, it’s a post-processor. It’s
easy enough to leverage PostCSS using npm scripts. Say we have a Sass
script like in our previous example:
"sass": "node-sass src/scss/ -o dist/css",
We can use npm script’s
lifecycle keywords to add a script to run automatically after the Sass task:
"postsass": "postcss --use autoprefixer -c postcss.config.json dist/css/*.css -d dist/css",
This script will run every time the Sass script is run. The postcss-cli package is great, because you can
specify configuration
in a separate file. Notice that in this example, we add another script
entry to accomplish a new task; this is a common pattern when using npm
scripts. You can create a workflow that accomplishes all of the various
tasks your app needs.
Conclusion
Task runners can solve real problems. I’ve used task runners to
compile different builds of a JavaScript application, depending on
whether the target was production or local development. I’ve also used
task runners to compile
Handlebars
templates, to deploy a website to production and to automatically add
vendor prefixes that are missed in my Sass. These are not trivial tasks,
but once they are wrapped up in a task runner, they became effortless.
Task runners are constantly evolving and changing. I’ve tried to
cover the most used ones in the current zeitgeist. However, there are
others that I haven’t even mentioned, such as
Broccoli,
Brunch and
Harp.
Remember that these are just tools: Use them only if they solve a
particular problem, not because everyone else is using them. Happy task
running!
