Skip to content

0.9.0
Compare
Choose a tag to compare
@svengreb svengreb released this 01 Apr 19:21
· 9 commits to main since this release
v0.9.0
This tag was signed with the committer’s verified signature.
svengreb Sven Greb
GPG key ID: 12637B9743D0263D
Learn about vigilant mode.
aac8bab

Changelog for the collection of template repositories for new projects.

Release Date: 2021-04-01 Project Board Milestone

⇅ Show all commits

Improvements

From npm to Yarn and back again — #72 ⇄ #73 (⊶ b996786)

↠ Some years ago, the switch from npm (v4) to Yarn (v1) was mainly done because of the fantastic workspace feature for monorepos as well as the great performance and UX improvements. This was a good decision and almost every popular and well-known project used to do the same, but with the announcement of Yarn v2 (named “berry“) the community got upset about the path the project has taken. Next to this, npm joined GitHub back in March 2020 which meant that the development continues in a good direction and is baked by the open source platform itself.
These events, the overall fantastic new npm release version v7, including the introduction of workspaces, and the fact that I never liked the disadvantage of requiring to use an “external“ package manager instead of the one that is bundled with Node, lead to the decision to finally switch back to npm again.

The only drawback is the constraint that the minimum npm version is now v7.7.0 because this is the first version that comes with workspace support for the run-script and exec commands through the --workspace/-w and --workspaces/-ws CLI flags, e.g. npm run -w PACKAGE run lint. The first Node version that ships with npm v7.7.x is v15.13.0 which is globally available as of april 1, 2021 (no, it‘s not an april fool 😄). To ensure that these constraints are met, without only relying on users to read the documentation, both npm and node have been added to the engines field of the package.json file.

This change also comes with updates to all documentations, including the addition of the version constraints, as well as updates to repository template features like the GitHub Action workflows. The .yarnrc file has been replaced by .npmrc that includes the package-lock=false and save-exact=false configurations.

Tasks

Node package dependency & GitHub action version updates — #67, #68, #69, #74 ⇄ #75

↠ Bumped outdated Go module dependencies and GitHub actions to their latest versions:

Dependency handling with lockfiles — #70 ⇄ #71 (⊶ a98d9b1)

↠ The usage of dependency lockfiles like package-lock.json or yarn.lock has always been a controversial topic where opinions go in different directions. On one side many project maintainers tend to argue that is helps to achieve deterministic build results, but on the side it might also hide problems when any later versions of a used dependency, or its transitive dependencies, is not compatible with the own project anymore.

I‘ve investigated a lot of time into research again to finally find a solution that works for my projects. In short, the result is to go with the rule that is also used by many large-scale projects: Do not use lockfiles for multi-consumer projects like libraries but only for single-consumer projects like applications.

Therefore the yarn.lock file has been removed since this makes no sense for a repository template anyway. See the sections below for some more details about how to decide to use a lockfile or not.

When to use lockfiles

The clear advantage of lockfiles are reproducible builds and the persistence of a running project state. They ensure that a project artifact can be rebuild at anytime using the exact same dependencies, resulting in the exact same artifact, even when the project was not updated in years.
This applies to projects that are focused on building a end-to-end experience like applications and other end-user products.

These are the advantages listed in the official npm documentation about package-lock.json files:

  • Describe a single representation of a dependency tree such that teammates, deployments, and continuous integration are guaranteed to install exactly the same dependencies.
  • Provide a facility for users to "time-travel" to previous states of node_modules without having to commit the directory itself.
  • Facilitate greater visibility of tree changes through readable source control diffs.
  • Optimize the installation process by allowing npm to skip repeated metadata resolutions for previously-installed packages.
  • As of npm v7, lockfiles include enough information to gain a complete picture of the package tree, reducing the need to read package.json files, and allowing for significant performance improvements.

Like mentioned, npm v7 comes with a lot of advantages and the team recommends to commit the file into project repositories:

  • the lockfile has enough information to describe the precise package tree all by itself.
  • the lockfile maps the packages to their information by their relative location to the root (instead of their name).
  • the npm CLI uses yarn.lock lockfiles if available, as a source of package metadata and resolution guidance when there is missing information, knowing that the package-lock.json is the authoritative definition.
    • yarn.lock lockfiles cannot completely replace npm’s lockfile since the current implementation doesn’t have enough information needed for the complete npm functionality.
  • the npm CLI uses a “hidden lockfile“ placed inside the node_module directory that helps to avoid repeated package tree reading.

Another point is that in end-user projects dependencies in package.json files are often pinned instead of using SemVer range selectors like ^ (latest minor-only) or ~ (latest patch only). In such cases a lockfile helps to keep control about transitive dependencies and persist projects states in time.

When to avoid lockfiles

Even though the Yarn team published a blog post in 2016 that states to always commit the yarn.lock file, regardless of the project type, this advice was not adopted by every project and some “real-world scenarios“ often showed that this decision was justified. There are blog posts that summarize when not to use a lockfile where even Yarn maintainers reply with comments that claim the opposite, but over the time more and more projects went away from using lockfiles.
One argument is that lockfiles are important to enure that library contributors in 10 years still know what was the last confirmed set of packages which worked as expected, but this can almost be ignored in a ecosystem like Node that changes almost every day.

Another important point is to mention that the usage of lockfiles were also a attack surface to inject malicious dependencies. Due to the large size of lockfiles, it is also often a challenge for project maintainers to review and validate a lockfile in pull requests are so they are often ignored and blindly trusted.

The community is still not of one opinion and I guess this will never change, but learning about the experience of well-known maintainers and popular projects is often a good way to find the own decision.

In conclusion, the usage of lockfiles in a non-end-user project can be well summarized with “just postponing the inevitable breakage“:

The full changelog is available in the repository

Copyright © 2020-present Sven Greb

Assets 2
Loading