Published agoric-cli package is missing most files

[mhofman]

Describe the bug

agoric-cli's package.json contains a files entry which is out of date with the content of the package. As a result, only the src/main.js ends up in the published package.

To Reproduce

Steps to reproduce the behavior:

1. yarn global add agoric@dev
2. Observe no global agoric bin
3. ls -laR ~/.config/yarn/global/node_modules/agoric/
4. Observe missing files

Expected behavior

A package containing all needed files

Platform Environment

• what OS are you using? what version of Node.js? Node 16, yarn 1.22.15
• is there anything special/unusual about your platform? Docker dev-env image
• what version of the Agoric-SDK are you using? dev

Additional context

We should have a test that exercises the content of the published packages to make sure all is ok with them

Screenshots

$ ls -laR ~/.config/yarn/global/node_modules/agoric/
/home/node/.config/yarn/global/node_modules/agoric/:
total 68
drwxr-xr-x   4 node node  4096 Dec 11 01:18 .
drwxr-xr-x 175 node node  4096 Dec 11 01:18 ..
-rw-r--r--   1 node node 32155 Dec 11 01:18 CHANGELOG.md
-rw-r--r--   1 node node 11357 Dec 11 01:18 LICENSE
-rw-r--r--   1 node node   835 Dec 11 01:18 README.md
drwxr-xr-x   3 node node  4096 Dec 11 01:18 node_modules
-rw-r--r--   1 node node  2126 Dec 11 01:18 package.json
drwxr-xr-x   2 node node  4096 Dec 11 01:18 src

/home/node/.config/yarn/global/node_modules/agoric/node_modules:
total 12
drwxr-xr-x 3 node node 4096 Dec 11 01:18 .
drwxr-xr-x 4 node node 4096 Dec 11 01:18 ..
drwxr-xr-x 2 node node 4096 Dec 11 01:18 .bin

/home/node/.config/yarn/global/node_modules/agoric/node_modules/.bin:
total 8
drwxr-xr-x 2 node node 4096 Dec 11 01:18 .
drwxr-xr-x 3 node node 4096 Dec 11 01:18 ..
lrwxrwxrwx 1 node node   33 Dec 11 01:18 opener -> ../../../opener/bin/opener-bin.js

/home/node/.config/yarn/global/node_modules/agoric/src:
total 16
drwxr-xr-x 2 node node 4096 Dec 11 01:18 .
drwxr-xr-x 4 node node 4096 Dec 11 01:18 ..
-rw-r--r-- 1 node node 7223 Dec 11 01:18 main.js

[mhofman]

One approach for the testing might be to do a actual publish to a test registry like https://verdaccio.org/ then do a yarn global add agoric@version, followed by a agoric install version-tag in a dapp.

[mhofman]

Even when fixing the files entry, apparently the bin/agoric symlink gets mangled when yarn pack and simply dropped when npm pack (see npm/npm#3310)

[mhofman]

Next up is @agoric/lockdown missing, but used by @agoric/install-ses. It's a peerDependency, so it doesn't get installed by default.

[mhofman]

And when manually adding @agoric/lockdown, I'm hitting the "missing name" error that @warner and @kriskowal have also experienced.

$ LOCKDOWN_OPTIONS='{"errorTaming":"unsafe","stackFiltering":"verbose"}'  agoric init hello
'@agoric/install-ses' sniffed and found a 'LOCKDOWN_OPTIONS' environment variable

Removing intrinsics.Object.hasOwn
(Error#1)
Error#1: missing name
  at Object.from (/home/node/.config/yarn/global/node_modules/ast-types/lib/types.js:270:27)
  at /home/node/.config/yarn/global/node_modules/ast-types/lib/types.js:236:71
  at Array.map (<anonymous>)
  at or (/home/node/.config/yarn/global/node_modules/ast-types/lib/types.js:236:37)
  at default_1 (/home/node/.config/yarn/global/node_modules/ast-types/def/core.js:264:25)
  at use (/home/node/.config/yarn/global/node_modules/ast-types/fork.js:45:31)
  at Array.forEach (<anonymous>)
  at Object.default_1 [as default] (/home/node/.config/yarn/global/node_modules/ast-types/fork.js:12:10)
  at Object.<anonymous> (/home/node/.config/yarn/global/node_modules/ast-types/main.js:21:24)
  at Module._compile (node:internal/modules/cjs/loader:1101:14)
  at Object.module.exports.o.a._extensions..js (/home/node/.config/yarn/global/node_modules/esm/esm.js:1:102412)
  at Module.load (node:internal/modules/cjs/loader:981:32)
  at Function.Module._load (node:internal/modules/cjs/loader:822:12)
  at Module.require (node:internal/modules/cjs/loader:1005:19)
  at require (node:internal/modules/cjs/helpers:102:18)
  at Object.<anonymous> (/home/node/.config/yarn/global/node_modules/@agoric/recast/main.js:6:34)
  at Module._compile (node:internal/modules/cjs/loader:1101:14)
  at Object.module.exports.o.a._extensions..js (/home/node/.config/yarn/global/node_modules/esm/esm.js:1:102412)
  at Module.load (node:internal/modules/cjs/loader:981:32)
  at Function.Module._load (node:internal/modules/cjs/loader:822:12)
  at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:190:29)
  at ModuleJob.run (node:internal/modules/esm/module_job:185:25)
  at async Promise.all (index 0)
  at async ESMLoader.import (node:internal/modules/esm/loader:281:24)
  at async loadESM (node:internal/process/esm_loader:88:5)
  at async handleMainPromise (node:internal/modules/run_main:65:12)

According to @kriskowal, workaround (for now) is to apply to following patch:

diff --git a/node_modules/ast-types/lib/types.js b/node_modules/ast-types/lib/types.js
index ce64cc1..ea6691e 100644
--- a/node_modules/ast-types/lib/types.js
+++ b/node_modules/ast-types/lib/types.js
@@ -266,10 +266,7 @@ function typesPlugin(_fork) {
                 if (bicfIndex >= 0) {
                     return builtInCtorTypes[bicfIndex];
                 }
-                if (typeof name !== "string") {
-                    throw new Error("missing name");
-                }
-                return new PredicateType(name, value);
+                return new PredicateType(name || value.name, value);
             }
             // As a last resort, toType returns a type that matches any value that
             // is === from. This is primarily useful for literal values like

[mhofman]

Next up, we're back in the agoric-cli land.

$ LOCKDOWN_OPTIONS='{"errorTaming":"unsafe","stackFiltering":"verbose"}'  agoric install beta
'@agoric/install-ses' sniffed and found a 'LOCKDOWN_OPTIONS' environment variable

Removing intrinsics.Object.hasOwn
agoric: install: yarn workspaces --silent info
agoric: install: yarn workspaces --silent info
warning package.json: No license field
error Cannot find the root of your workspace - are you sure you're currently in a workspace?
agoric: (SyntaxError#1)
SyntaxError#1: Unexpected end of JSON input
  at JSON.parse (<anonymous>)
  at workspaceDirectories (file:///home/node/.config/yarn/global/node_modules/agoric/src/install.js:37:20)
  at processTicksAndRejections (node:internal/process/task_queues:96:5)
  at async installMain (file:///home/node/.config/yarn/global/node_modules/agoric/src/install.js:219:19)
  at async Promise.all (index 0)
  at async main (file:///home/node/.config/yarn/global/node_modules/agoric/src/main.js:237:5)

This is on a dapp just created by agoric init.

Cause of the failure is:

agoric-sdk/packages/agoric-cli/src/install.js

Lines 218 to 219 in e852ee5

	const sdkRoot = path.resolve(dirname, `../../..`);
	const sdkDirs = await workspaceDirectories(sdkRoot);

Admittedly I have no idea why we're looking up 3 directories?

[mhofman]

Removing that chunk of code allows agoric-cli install beta to proceed, but then it doesn't seem to do what I'd expect, see #4170