Using nodejs for everything
When developing a fairly complex web application, one of the things I hate most is how mangled and tangled the 'scripts' section of package.json becomes over time. There has to be a better way.
Open any "well-seasoned" fullstack nodejs web application, and take a look at its scripts
section inside the package.json file. What a mess, isn't it?
- You can't even add comments, because json right?
- Who's using these scripts? When? In which phase of development?
- they cannot be tested (obviously)
- No dependency tree, so good luck figuring out what needs to run and when
- Check the project's README to see if it's updated with that scripts section (spoiler: it's not)
- Oh, did you consider developers running Windows? Oops...
- And what about
NODE_ENV
? Where is it set? How does that affect these scripts?
It's a mess, and I hate it. But we also need scripts, right? In my most recent pet project, I wanted to decouple the build process and use as few dependencies as possible. That, of course, created the side effect of having to manually orchestrate several steps like building, linting, restarting and so on.
For the record, the project itself uses Fastify, JSX (server-side rendered, no React), fairly light client-side JS (with AlpineJS and HTMX), esbuild and TypeScript. It has been created on top of my own "starter kit" for HTMX and Fastify.
This is what I ended up with in my scripts
:
"dev": "npm run clean && conc -r -k 'npm:dev:*'",
"dev:start": "wait-on ./dist/server/app.js && node --env-file=.env --watch --watch-preserve-output --watch-path ./dist/server ./dist/server/app.js",
"dev:build": "npm run lint && node --env-file=.env esbuild.mjs",
"build": "npm run lint && NODE_ENV=production node --env-file=.env esbuild.mjs",
"clean": "rm -rf ./dist",
"type-check": "tsc -p server && tsc -p client",
"lint": "npm run type-check && biome lint .",
"start": "NODE_ENV=production node --env-file=.env ./dist/server/app.js"
...
It is ridicolous, not to mention all the "helpers" I had to install, like concurrency
and wait-on
, or rimraf
.
This is actually a problem that's been solved since the very beginning of the "bundling era" in the frontend land - that's what task runners like grunt
, gulp
, rake
, yeoman
, or more recently even turborepo
and nx
are for. The newer ones also want to improve the development experience when working on big and complex monorepos.
But here's the thing: when you decide to use one of those tools (which isn't even à la mode anymore these days), you're adding additional dependencies. More often than not, these tools use plugins that need to be installed, and of course you need to learn how to effectively use them. I can't remember how many times I ended up rewriting my old gulpfiles for the sake of performance or clarity, or whatever else I needed to do.
What could be another solution then? Well, why not just use Node.js to its full potential? Nowadays with Node.js, without installing a single dependency, you can watch files for changes, start simple HTTP servers, read a .env file, and of course spawn subprocesses - all in a cross-platform way!
And this is what I did, writing a (huge) script that I called "hawk" which does everything without much orchestration needed in the package.json file. And what is everything, you ask? Well:
- Cleans the output directory
- Runs type checking for the client and server (using TypeScript --noEmit)
- Runs the linter for the client and the server (using Biome)
- Builds the client and server (using esbuild)
- Starts a SSE event to notify the client when a change happens (a new build)
- Sets up file watchers to run all these tasks in proper order, depending on what changed
- Starts and restarts the API server (the Fastify backend)
And all of this only using Node.js and the packages that I would need anyway in the project (esbuild, ts, and biome). This is the important bit for me: no additional dependencies, all the logic in one place, and all the logic in TypeScript.
The only exception to that rule is the usage of tsx
that I need because I want hawk
to be written in TS too.
Now my scripts
sections looks like this:
"dev": "NODE_ENV=development tsx --env-file=.env hawk.ts",
"build": "NODE_ENV=production tsx --env-file=.env hawk.ts",
"start": "NODE_ENV=production node --env-file=.env ./dist/server/app.js"
If you are curious about the script (which is still a work in progress, honestly) take a look at the file in the repo.
And if you really want the TL;DR, this is how my script ends:
const taskManager = new TaskManager();
taskManager.registerTask('clean-client', clean, 'client');
taskManager.registerTask('clean-server', clean, 'server');
taskManager.registerTask('type-check-client', typeCheck, 'client');
taskManager.registerTask('type-check-server', typeCheck, 'server');
taskManager.registerTask('lint-client', lint, 'client');
taskManager.registerTask('lint-server', lint, 'server');
taskManager.registerTask('build-client', buildClient);
taskManager.registerTask('build-server', buildServer);
taskManager.registerTask('start-sse-server', startSseServer);
taskManager.registerTask('setup-file-watchers', setupFileWatchers);
taskManager.registerTask('start-api-server', startApiServer);
taskManager.registerTask('ready', ready);
taskManager.registerTask('long-task', longTask);
taskManager.registerTask('notify-client-update', notifyUpdates, 'client');
taskManager.registerTask('notify-server-update', notifyUpdates, 'server');
if (process.env.NODE_ENV === 'production') {
taskManager.run(
['type-check-client', 'build-client', 'type-check-server', 'build-server'],
true
);
} else {
taskManager.run(
[
'clean-client',
'clean-server',
'type-check-client',
'lint-client',
'build-client',
'type-check-server',
'lint-server',
'build-server',
'start-sse-server',
'setup-file-watchers',
'start-api-server',
'ready',
],
true
);
}
What's next with this idea? This is something I did because I needed it, and I needed it fast; now I can improve the idea:
- use the TypeScript API directly instead of spawning the
tsc
process for type checking - use the Biome API for the same reason, but they its API is still unstable
- maybe try to package it somehow, more as a collection of "practices" than an actually tool, to whomever wants to pursue the same approach
Thank you for reading!