build: automate release process

[dsebastien]

We should have a script that helps us doing a release easily and automatically.
It should

• update package.json (semver)
• update changelog
• tag
• push the tag
• let travis handle the npm publish

First

• review what angular does: https://github.com/angular/angular/tree/master/scripts/release
• evaluate semantic-release
• evaluate standard-version

[dsebastien]

Related to #31

[dsebastien]

Related to #24: semantic-release follows Angular's commit message guidelines

[dsebastien]

First proposal:

• pre-reqs (travis)
  • NPM_TOKEN environment variable
  • or npm login against official npm registry?
• clean:all
  • remove node_modules for each package
  • remove package-lock.json for each package
  • remove node_modules at root
  • remove package-lock.json at root
• cache verify
• update root package.json version (semantic-release?)
• install:all (root + all modules)
• commit package-lock.json (root + all modules)
• build:all
• update changelog
• commit changelog
• git tag version + publish changelog (releases page on GH)
• publish each module on npm
• announcement
  • twitter
  • internal NBB

[SuperITMan]

mmm...
It should be :

• set root package.json version
• install
• install:all
• commit package-lock.json

[dsebastien]

Second proposal after internal discussion:

First initiate the release manually through: npm run release; which will

• make sure there are no local changes
• execute release-it: https://github.com/webpro/release-it
  • updates the root package.json version automatically
    • that version number will be used as basis in the build to adapt all other package.json files
  • git add package.json
  • update changelog: conventional-changelog used by release-it
  • git add changelog
  • git commit -m 'chore(release): release prepare'
  • git tag
• git push --follow-tags
• git create release page on GH: https://github.com/conventional-changelog/releaser-tools

Once done, a Travis build will be triggered.

Travis pre-reqs:

• dedicated npm account!
• NPM_TOKEN environment variable
  • auth-only level for two-factor auth (cfr https://docs.npmjs.com/getting-started/using-two-factor-authentication#levels-of-authentication)

In the travis build, the usual steps will be performed:

• npm run lint:all
• npm run test:ci:all

Travis initial config and those scripts take care of these:

• clean:all
  • remove node_modules for each package
  • remove package-lock.json for each package
  • remove node_modules at root
  • remove package-lock.json at root
• cache verify
• install:all (root + all modules)
• build:all
• test:all

Then an additional script, will be executed: npm run release:publish will be executed, which will:

• check if a release is necessary by checking if the last commit message starts with chore(release): release or that Travis is executing for a tag: if [ -n "$TRAVIS_TAG" ]; then something when tag set; fi
  • if not, it stops there
  • the message is accessible as TRAVIS_COMMIT_MESSAGE (other vars: https://docs.travis-ci.com/user/environment-variables/)
  • or it can be retrieved with git: git log --format=%B -n 1
  • check with if grep -q 'chore(release): release'; then ... fi;
• publish each module on npm

Finally we need to make announcements:

• twitter
• internal NBB

Questions:

• npm tags?
  • latest
  • how to set those?

[dsebastien]

Issues to fix

1. with the above, once release:publish runs on Travis, we use the result of he previous commands that did npm run install:all which also calls npm run build but WITHOUT passing the --publish option, meaning that the version in the package.json files will contain the last commit hash.

We need figure out how to remove that suffix when doing the release

One option:

• split release:perform script
  • release-prechecks, build with --publish, release
  • if release-prechecks don't pass then the rest should just be skipped
  • issues
    • doubles build time for releases since we build a second time
    • risk of stale files
• extract logic to adapt the package.json of all packages
  • call that logic again in the release script
  • probably also means isolating the list of packages

2. should we remove the package-lock.json files during the release since they've just been updated by the build? Problem is if the release is done locally without travis. This will change when we go back to npm ci...

[dsebastien]

Useful angular scripts:

angular publish: https://github.com/angular/angular/blob/master/scripts/release/publish
#!/usr/bin/env bash

# Use for PATCH releases
# Publish all packages in `dist/packages-dist` to npm

(cd dist/packages-dist; for p in `ls .`; do npm publish --access public $p; done)



angular publish-next: https://github.com/angular/angular/blob/master/scripts/release/publish-next
#!/usr/bin/env bash

# Use for BETA and RC releases
# Publish all packages in `dist/packages-dist` to npm (as next)

(cd dist/packages-dist; for p in `ls .`; do npm publish --access public --tag next $p; done)



angular pre-check: https://github.com/angular/angular/blob/master/scripts/release/pre-check

#!/usr/bin/env bash

# Verify peer deps constraints and package.json before publishing to npm
# There should be no npm errors

mkdir tmp
cd tmp
npm init -y
npm install --save ../dist/packages-dist/* zone.js@0.8.10 rxjs@6.0.0-beta.1 typescript@2.7
cd ..
rm -rf ./tmp



angular post-check: https://github.com/angular/angular/blob/master/scripts/release/post-check


#!/usr/bin/env bash

# use for PATCH releases
# Verify that all packages are published with correct version:

mkdir tmp
cd tmp
npm init -y
npm install typescript@2.6 rxjs@5.5.0 zone.js@0.8.10 @nationalbankbelgium/{stark-build,stark-core,stark-testing} --save
cd ..
rm -rf tmp

echo "CHECK THE DIST-TAGS"

grep '"name": "@nationalbankbelgium' $(find packages -name package.json) | grep -o '@nationalbankbelgium/[^/"]*' | sort -u | xargs -n 1 -I% npm show \% name dist-tags




angular post-check-next: https://github.com/angular/angular/blob/master/scripts/release/post-check-next



#!/usr/bin/env bash

# use for BETA and RC releases
# Verify that all packages are published with correct version:

mkdir tmp
cd tmp
npm init -y
npm install typescript@2.7 rxjs@6.0.0-beta.1 zone.js@0.8.20 @nationalbankbelgium/{stark-build,stark-core,stark-testing}@next --save
cd ..
rm -rf tmp

echo "CHECK THE DIST-TAGS"

grep '"name": "@angular' $(find packages -name package.json) | grep -o '@nationalbankbelgium/[^/"]*' | sort -u | xargs -n 1 -I% npm show \% name dist-tags

[dsebastien]

Proposal needs review.
If we use standard-version, then we should let it do bump, changelog, commit AND tag.
If the tag is created and pushed, then the client should also handle the github release page creation.

On Travis

• we should not check the commit message as proposed before, but the fact that the build was triggered by a tag creation
• the package-lock.json files add/commit should be forgotten for now ??

[dsebastien]

I've configured the NPM_TOKEN variable in Travis.
NEVER log that token ;-)

[dsebastien]

One issue to be aware of: release-it/release-it#273

[dsebastien]

For the tags like nightly, dev, beta and the like, we could have additional scripts passing an argument for that