Why not commit lockfiles? (package-lock.json) #2409
Labels
Comments
|
Only apps should have lockfiles. Dev-only lockfiles (package-lock.json and yarn.lock) achieve nothing in a package except artificially insulating maintainers from problems their users will experience. It's thus hostile to a package's consumers to use a lockfile. There is no "general consensus" to do so, there's mostly just "npm's default behavior" which the majority cargo-cults. |
|
@ljharb Thanks for answering, but I'd just like to clarify what you're saying here: Because the only packages declared in |
|
No, that's not what I'm saying. I'm saying that package-lock doesn't get published - so, consumers of a package don't "get" that lockfile. So, consumers will get the latest version of both direct and transitive dependencies - but the package was only tested with "whatever is in its lockfile", so the package's tests might be passing while the consumers' code is broken. |
|
@ljharb And how are misleading CI test results and broken consumer code good? ^^ |
|
They're not - that's what "having a lockfile" gets you. |
svengreb
added a commit
to svengreb/tmpl
that referenced
this issue
Apr 1, 2021
The usage of dependency lockfiles like `package-lock.json` [1] or
`yarn.lock` [2] has always been a controversial topic where opinions go
in different directions. On one side many project maintainers tend to
argue that is helps to achieve deterministic build results, but on the
side it might also hide problems when any later versions of a used
dependency, or its transitive dependencies, is not compatible with the
own project anymore.
I've investigated a lot of time into research again to finally find a
solution that works for my projects. In short, the result is to go with
the rule that is also used by many large-scale projects:
Do not use lockfiles for multi-consumer projects like libraries but only
for single-consumer projects like applications.
Therefore the `yarn.lock` file has been removed since this makes no
sense for a repository template anyway. See the sections below for some
more details about how to decide to use a lockfile or not.
>>> When to use lockfiles
The clear advantage of lockfiles are reproducible builds and the
persistence of a running project state. They ensure that a project
artifact can be rebuild at anytime using the exact same dependencies,
resulting in the exact sFame artifact, even when the project was not
updated in years.
This applies to projects that are focused on building a end-to-end
experience like applications and other end-user products.
These are the advantages listed in the official npm documentation about
`package-lock.json` files:
"- Describe a single representation of a dependency tree such that
teammates, deployments, and continuous integration are guaranteed
to install exactly the same dependencies.
- Provide a facility for users to "time-travel" to previous states of
`node_modules` without having to commit the directory itself.
- Facilitate greater visibility of tree changes through readable
source control diffs.
- Optimize the installation process by allowing npm to skip repeated
metadata resolutions for previously-installed packages.
- As of npm `v7`, lockfiles include enough information to gain a
complete picture of the package tree, reducing the need to read
`package.json` files, and allowing for significant performance
improvements."
Like mentioned, npm `v7` comes with a lot of advantages and the team
recommends to commit the file into project repositories [3]:
- the lockfile has enough information to describe the precise package
tree all by itself.
- the lockfile maps the packages to their information by their relative
location to the root (instead of their name).
- the npm CLI uses `yarn.lock` lockfiles if available, as a source of
package metadata and resolution guidance when there is missing
information, knowing that the `package-lock.json` is the authoritative
definition.
- `yarn.lock` lockfiles cannot completely replace npm’s lockfile since
the current implementation doesn’t have enough information needed
for the complete npm functionality.
- the npm CLI uses a "hidden lockfile" placed inside the `node_module`
directory that helps to avoid repeated package tree reading.
Another points is that in end-user projects dependencies in
`package.json` files are pinned often instead of using SemVer range
selectors [4] like `^` (latest minor-only) or `~` (latest patch only).
In such cases a lockfile helps to keep control about transitive
dependencies and persist projects states in time.
>>> When to avoid lockfiles
Even though the Yarn team published a blog post in 2016 [5] that states
to always commit the `yarn.lock` file, regardless of the project type,
this advice was not adopted by every project and some
"real-world scenarios" often showed that this decision was justified.
There are blog posts that summarize when not to use a lockfile [6] where
even Yarn maintainers reply with comments that claim the opposite [7],
but over the time more and more projects went away from using lockfiles.
One argument is that lockfiles are important to enure that library
contributors in 10 years still know what was the last confirmed set of
packages which worked as expected [8], but this can almost be ignored in
a ecosystem like Node that changes almost every day.
Another important point is to mention that the usage of lockfiles were
also a attack surface to inject malicious dependencies [9]. Due to the
large size of lockfiles, it is also often a challenge for project
maintainers to review and validate a lockfile in pull requests are so
they are often ignored and blindly trusted [10].
The community is still not of one opinion and I guess this will never
change, but learning about the experience of well-known maintainers [11]
and popular projects [12] is often a good way to find the own decision.
In conclusion, the usage of lockfiles in a non-end-user project can be
well summarized with "just postponing the inevitable breakage" [13].
[1]: https://docs.npmjs.com/cli/v7/configuring-npm/package-lock-json
[2]: https://classic.yarnpkg.com/en/docs/yarn-lock
[3]: https://blog.npmjs.org/post/621733939456933888/npm-v7-series-why-keep-package-lockjson.html
[4]: https://semver.npmjs.com
[5]: https://classic.yarnpkg.com/blog/2016/11/24/lockfiles-for-all
[6]: https://dev.to/gajus/stop-using-package-lock-json-or-yarn-lock-3ddi
[7]: https://dev.to/arcanis/comment/fo33
[8]: https://twitter.com/arcanis/status/1164229994165559299?s=19
[9]: https://snyk.io/blog/why-npm-lockfiles-can-be-a-security-blindspot-for-injecting-malicious-modules
[10]: https://twitter.com/bcrypt/status/1208950722097598465
[11]: sindresorhus/ama#479 (comment)
[12]: airbnb/javascript#2409
[13]: https://twitter.com/renovatebot/status/1163789817492230144
GH-70
svengreb
added a commit
to svengreb/tmpl
that referenced
this issue
Apr 1, 2021
The usage of dependency lockfiles like `package-lock.json` [1] or
`yarn.lock` [2] has always been a controversial topic where opinions go
in different directions. On one side many project maintainers tend to
argue that is helps to achieve deterministic build results, but on the
side it might also hide problems when any later versions of a used
dependency, or its transitive dependencies, is not compatible with the
own project anymore.
I've investigated a lot of time into research again to finally find a
solution that works for my projects. In short, the result is to go with
the rule that is also used by many large-scale projects:
Do not use lockfiles for multi-consumer projects like libraries but only
for single-consumer projects like applications.
Therefore the `yarn.lock` file has been removed since this makes no
sense for a repository template anyway. See the sections below for some
more details about how to decide to use a lockfile or not.
>>> When to use lockfiles
The clear advantage of lockfiles are reproducible builds and the
persistence of a running project state. They ensure that a project
artifact can be rebuild at anytime using the exact same dependencies,
resulting in the exact sFame artifact, even when the project was not
updated in years.
This applies to projects that are focused on building a end-to-end
experience like applications and other end-user products.
These are the advantages listed in the official npm documentation about
`package-lock.json` files:
"- Describe a single representation of a dependency tree such that
teammates, deployments, and continuous integration are guaranteed
to install exactly the same dependencies.
- Provide a facility for users to "time-travel" to previous states of
`node_modules` without having to commit the directory itself.
- Facilitate greater visibility of tree changes through readable
source control diffs.
- Optimize the installation process by allowing npm to skip repeated
metadata resolutions for previously-installed packages.
- As of npm `v7`, lockfiles include enough information to gain a
complete picture of the package tree, reducing the need to read
`package.json` files, and allowing for significant performance
improvements."
Like mentioned, npm `v7` comes with a lot of advantages and the team
recommends to commit the file into project repositories [3]:
- the lockfile has enough information to describe the precise package
tree all by itself.
- the lockfile maps the packages to their information by their relative
location to the root (instead of their name).
- the npm CLI uses `yarn.lock` lockfiles if available, as a source of
package metadata and resolution guidance when there is missing
information, knowing that the `package-lock.json` is the authoritative
definition.
- `yarn.lock` lockfiles cannot completely replace npm’s lockfile since
the current implementation doesn’t have enough information needed
for the complete npm functionality.
- the npm CLI uses a "hidden lockfile" placed inside the `node_module`
directory that helps to avoid repeated package tree reading.
Another points is that in end-user projects dependencies in
`package.json` files are pinned often instead of using SemVer range
selectors [4] like `^` (latest minor-only) or `~` (latest patch only).
In such cases a lockfile helps to keep control about transitive
dependencies and persist projects states in time.
>>> When to avoid lockfiles
Even though the Yarn team published a blog post in 2016 [5] that states
to always commit the `yarn.lock` file, regardless of the project type,
this advice was not adopted by every project and some
"real-world scenarios" often showed that this decision was justified.
There are blog posts that summarize when not to use a lockfile [6] where
even Yarn maintainers reply with comments that claim the opposite [7],
but over the time more and more projects went away from using lockfiles.
One argument is that lockfiles are important to enure that library
contributors in 10 years still know what was the last confirmed set of
packages which worked as expected [8], but this can almost be ignored in
a ecosystem like Node that changes almost every day.
Another important point is to mention that the usage of lockfiles were
also a attack surface to inject malicious dependencies [9]. Due to the
large size of lockfiles, it is also often a challenge for project
maintainers to review and validate a lockfile in pull requests are so
they are often ignored and blindly trusted [10].
The community is still not of one opinion and I guess this will never
change, but learning about the experience of well-known maintainers [11]
and popular projects [12] is often a good way to find the own decision.
In conclusion, the usage of lockfiles in a non-end-user project can be
well summarized with "just postponing the inevitable breakage" [13].
[1]: https://docs.npmjs.com/cli/v7/configuring-npm/package-lock-json
[2]: https://classic.yarnpkg.com/en/docs/yarn-lock
[3]: https://blog.npmjs.org/post/621733939456933888/npm-v7-series-why-keep-package-lockjson.html
[4]: https://semver.npmjs.com
[5]: https://classic.yarnpkg.com/blog/2016/11/24/lockfiles-for-all
[6]: https://dev.to/gajus/stop-using-package-lock-json-or-yarn-lock-3ddi
[7]: https://dev.to/arcanis/comment/fo33
[8]: https://twitter.com/arcanis/status/1164229994165559299?s=19
[9]: https://snyk.io/blog/why-npm-lockfiles-can-be-a-security-blindspot-for-injecting-malicious-modules
[10]: https://twitter.com/bcrypt/status/1208950722097598465
[11]: sindresorhus/ama#479 (comment)
[12]: airbnb/javascript#2409
[13]: https://twitter.com/renovatebot/status/1163789817492230144
Closes GH-70
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
I just noticed that you guys have decided to not commit the npm lockfile
package-lock.json, although the general consensus is to do so. I'm curious why you have done that, as I don't really understand the purpose of it either.The text was updated successfully, but these errors were encountered: