Ryan's Tech Blog

I’ve been working with javascript and AngularJS a lot recently, both at work and in hobby projects. I’m a big fan of the framework, but like most non-trivial javascript frameworks, it really wants to have a build/compile step. There are a lot of options for javascript build tools. I identified some main contenders: npm, grunt, gulp, browserify, webpack, bower, requirejs.

Recently I’ve been experimenting with just using npm and browserify, and wanted to summarize my results.

TL;DR: works well for small projects, but I think I need to add something like grunt or gulp (g(runt|ulp)).

Goals

The benefits I’m looking to get from my build tools:

• can use many small files: angularjs code is easier for me to write and test if I have many small js/html files. Serving many small files to users is bad for performance (lots of HTTP requests), and I’m bad at maintaining a curated list of script tags for what to serve. Ideally I’m serving one or two files that contain everything my app needs
• can use third-party libs: there are lots of good open source libraries that I want to use. I’m bad at managing those dependencies (and their dependencies) by hand, and I feel bad committing a minified library into source control

My test was a small web app to track when the last time I did a house chore: when-did-i. The source is up at ryepup/when-did-i. I’m actually experimenting with a bunch of different stuff, but I’m only going to consider npm and browserify here.

Using npm for third-party dependencies

Most libraries are published to npm, and I never had an issue with missing a library, and was able to keep all external libs out of my repo.

The only weird thing is the version specifier in package.json. By default, if you install package X (npm install --save X), it’ll find the current version (say 1.2.3), and then add it to package.json with a version specifier like ^1.2.3. This basically means “1.2.3” or anything newer with a 1.x.y version. This can cause some surprises, especially if you have a continuous integration setup. Your CI robot might be testing different versions than what you are developing against.

The solution to this is npm shrinkwrap, to specify precisely every version of every piece of software you want. It’s basically the equivalivent of python’s pip freeze and requirements.txt.

Using npm scripts for build actions

This worked out well… up to a point. Using npm scripts gave me easy access to a lot of npm installed command line programs, without needing to install them globally on my system or muck about with my path. I like keeping the project’s needs self-contained. There are a ton of small tools available on npm to do just about anything.

I like the simplicity; there’s no explicit “target” like other build tools, you just have a name, and what command you want to run with node’s path all setup. Then you can say npm run $NAME and it’ll go. You can add a “pre” or “post” prefix to the name to run other commands before/after. If you want to call your other scripts, you just use npm run my-other-script as part of your command. Pretty easy, pretty basic.

The problem arises when you want to do something more complex. The worst one I had was start:

"prestart": "npm install",
"start": "watch 'npm run build' src/ & live-reload --port 9091 ./build/* & ws -d ./build",

Let’s break it down:

1. a “pre” script to ensure packages are installed first if someone runs npm run start
2. start the npm-installed watch command to look file changes in src/, and run npm run build when something changes (this is an example of one command calling another), then we have an & to run watch in the background – this means our “start” script doesn’t work on windows
3. start the npm-installed live-reload to run a LiveReload server on localhost:9091 to refresh my browser when something changes in build/, and another & to run this in the background
4. start the npm-installed ws web server to serve the files in my build folder at localhost:8000

With that combination, as I edit my files they get rebuilt and my browser refreshes.

This is a pretty standard frontend development workflow, and I feel like it’s too much to squeeze into a one liner. I could make some short nodejs scripts that launch these services, but at that point I feel like I’m reinventing a wheel and I should just pull in g(runt|ulp).

Using browserify to combine files

I think this worked out pretty well, but also had some quirks. By using require statements, I was able to centralize all my angularjs registrations into one file, which felt really nice and reduced some boilerplate. Each of my javascript files was basically defining one function. I really liked not having to manually specify an IIFE in each file. It also generates all the source maps, so debugging in the browser is referring to my small files, not whatever the bundle produces.

browserify has a pretty rich plugin system, and I used browserify-ng-html2js to support keeping my templates in separate html files. This is another place where npm scripts broke down a little. By default browserify-ng-html2js puts each html file into it’s own angular module, and then I need to make my main angular module depend on each individual template. This is back to a manually curated list that I’m going to screw up. browserify-ng-html2js has an option to put all the templates into one module, but that only seems to be available if you use g(runt|ulp).

Pulling everything in via npm means I could have one bundled file that contains my code and all it’s dependencies. This gets to be kinda a big file. I added some machinery to reference some angularjs libs from a CDN, but the easist path with browserify is to have everything just included. I guess if you’re using cache headers well and versioning in the URL this might be alright. Rigth now I’m at 264KB (73.4KB over the wire) which does include some dependencies. Letting browserify combine ALL my dependencies would more than double the file size. I’m really not sure if that matters, but it makes me nervous.

In the past I’ve used some grunt machinery to maintain the list of scripts to load; I liked this a little better because what I was developing with was closer to what I’m deploying.

Conclusion

I like the browserify and npm combination, but npm scripts are too limited, and another build tool is required. I think npm scripts are good enough as a task runner for simple dev or CI, but build steps just need more configration. It’s possible that maybe the specific build libraries could better support looking at package.json for configuration, but there’s just a lot more momentum behind using g(runt|ulp).