TIL: How NPM Lockfiles Actually Work
And why your CI/CD should use npm ci
TIL that I’ve been misunderstanding how npm lockfiles work!
I always thought npm install
would automatically update to the latest versions within your semver ranges, but that’s not quite right. Though it was plausibly right for old versions of npm but changed a few years ago…
How NPM Install Actually Works
Here’s the key insight I missed: npm install
will only update packages if your package.json
doesn’t match your package-lock.json
.
So if you have:
package.json
says:"react": "^18.0.0"
package-lock.json
has:"react": "18.2.0"
Then npm install
will use 18.2.0
from the lockfile, not go fetch 18.3.1
(or whatever the latest 18.x version is).
The problems only happen when:
- Someone modifies
package.json
but doesn’t commit the updated lockfile - The lockfile is missing entirely
- There’s a mismatch between the two files
🚨 The Hidden Problem: CI/CD Misconfigurations
As a consultant, I’ve seen this pattern way too many times:
Jenkins pipeline or GitHub Actions doing:
npm install
npm run build
Instead of:
npm ci
npm run build
This is a huge difference!:
npm install
will try to “fix” any mismatches between package.json and lockfilenpm ci
will fail fast if there are any mismatchesnpm ci
is safest because it uses the lockfile exactly as-is, ensuring reproducible builds
War Stories
I’ve debugged so many “works on my machine” issues that trace back to:
- Add a package locally:
npm install --save lodash
(or commited package.json by hand editing and not with the cli npm install command) - commits
package.json
but missed committingpackage-lock.json
- CI/CD runs
npm install
and gets a different version of lodash than we did - Production breaks because of subtle differences 😬
Or:
- CI/CD uses
npm install
instead ofnpm ci
- New vulnerability gets published in a dependency
- CI/CD automatically pulls the latest “safe” version with the vulnerability because lockfile was out of sync
- Security incident happens
My workflow today
Initial project clone:
- Use
npm ci
to verify lockfile is correct and modules are expected
For Local Dev:
- Use
npm ci
except if I know I want to modify dependencies, out of habit - Use
npm install --save
if I want to add/update a specific package - Always commit both
package.json
ANDpackage-lock.json
together
For CI/CD Pipelines:
- Always use
npm ci
instead ofnpm install
- This guarantees your build uses exactly what’s in the lockfile
- It fails fast if someone forgot to commit the lockfile
🤝 Building on Great Ideas
The recent CrowdStrike-themed npm supply chain attack is a perfect example of why lockfiles matter in the npm world today. But there are other ideas on how to improve it all:
This whole learning journey was sparked by some timely blogs. Jim Nielsen’s post about running software combinations that have never been tested together really resonated. And Niki@tonsky’s deep dive into how Maven/Gradle have worked for years without lockfiles in Java land.
The Maven approach of deterministic dependency resolution is arguably cleaner than the NPM lockfile system. But until the npm ecosystem evolves, we need to use the tools we have correctly.
🚀 TL;DR for your CI/CD
- Use
npm ci
instead ofnpm install
- Always commit both
package.json
andpackage-lock.json
- lockfiles are a necessary evil (even if they’re confusing)
I’m sure I’m wrong in here somewhere, but at least now I documented how i think it all works so I can re-read this later when I learn I’m wrong again 😅
Side note: My original confused idea of npm ci - on Mastodon