feat(cli): evaluate code snippets in JSDoc and markdown

[magurotuna]

This commit lets deno test --doc command actually evaluate code snippets in JSDoc and markdown files.

How it works

1. Extract code snippets from JSDoc or code fences
2. Convert them into pseudo files by wrapping them in Deno.test(...)
3. Register the pseudo files as in-memory files
4. Run type-check and evaluation

We apply some magic at the step 2 - let's say we have the following file named mod.ts as an input:

/**
 * ```ts
 * import { assertEquals } from "jsr:@std/assert/equals";
 * 
 * assertEquals(add(1, 2), 3);
 * ```
 */
export function add(a: number, b: number) {
  return a + b;
}

This is virtually transformed into:

import { assertEquals } from "jsr:@std/assert/equals";
import { add } from "files:///path/to/mod.ts";

Deno.test("mod.ts$2-7.ts", async () => {
  assertEquals(add(1, 2), 3);
})

Note that a new import statement is inserted here to make add function available. In a nutshell, all items exported from mod.ts become available in the generated pseudo file with this automatic import insertion.

The intention behind this design is that, from library user's standpoint, it should be very obvious that this add function is what this example code is attached to. Also, if there is an explicit import statement like import { add } from "./mod.ts", this import path ./mod.ts is not helpful for doc readers because they will need to import it in a different way.

The automatic import insertion has some edge cases, in particular where there is a local variable in a snippet with the same name as one of the exported items. This case is addressed by employing swc's scope analysis (see test cases for more details).

"type-checking only" mode stays around

This change will likely impact a lot of existing doc tests in the ecosystem because some doc tests rely on the fact that they are not evaluated - some cause side effects if executed, some throw errors at runtime although they do pass the type check, etc.
To help those tests gradually transition to the ones runnable with the new deno test --doc, we will keep providing the ability to run type-checking only via deno check --doc. Additionally there is a --doc-only option added to the check subcommand too, which is useful when you want to type-check on code snippets in markdown files, as normal deno check command doesn't accept markdown.

Demo

deno_test_doc_demo.mp4

---

Closes #4716

[bartlomieju]

@magurotuna this looks great, can we use it to test our own declarations files (eg. cli/tsc/dts/lib.deno.ns.d.ts)?

[magurotuna]

can we use it to test our own declarations files (eg. cli/tsc/dts/lib.deno.ns.d.ts)?

I did (against deno_std as well), and what turned out was some fail because they are not self-contained and make some assumption on the testing environment that is not always the case, or some even hung - which made me feel we need to preserve the existing type-checking only mode in another command 😄

Here's the result of running deno test --doc cli/tsc/dts/lib.deno.ns.d.ts.
https://gist.github.com/magurotuna/ec370702205d5c7262f1aeae397ef184

This says that the test case starting from line 2781 hangs - and actually this test case is waiting for something from stdin, which obviously doesn't occur, thus resulting in unfinished:

deno/cli/tsc/dts/lib.deno.ns.d.ts

Lines 2781 to 2787 in c29e5b9

	* ```ts
	* const decoder = new TextDecoder();
	* for await (const chunk of Deno.stdin.readable) {
	* const text = decoder.decode(chunk);
	* // do something with the text
	* }
	* ```

And one of the failing test cases is this, where it tries to open my_file.txt but generally this file does not exist in the testing environment.

deno/cli/tsc/dts/lib.deno.ns.d.ts

Lines 2725 to 2728 in c29e5b9

	* ```ts
	* using file = await Deno.open("my_file.txt");
	* // do work with "file" object
	* ```

So I think we can conclude that the new doc test feature is working as we expect, although many of the existing example code snippets in JSDoc need to be updated.

[iuioiua]

Two blocking concerns, IMO:

1. Does the execution of code snippets respect permissions?
2. Is there a way to skip the evaluation of a particular code snippet? std uses a no-eval directive.

[magurotuna]

Does the execution of code snippets respect permissions?

Yes it does. I added another test case to illustrate it 13235e2. Let's clarify this in the document later.

Is there a way to skip the evaluation of a particular code snippet? std uses a no-eval directive.

The existing directive ignore will keep working. Code snippets annotated with ignore will not be type-checked or evaluated when we run deno test --doc.

[bartlomieju]

LGTM, this is a great first pass and we can keep iterating on this one. @magurotuna could you please update the docs page at https://docs.deno.com/runtime/fundamentals/testing/?

[magurotuna]

could you please update the docs page at https://docs.deno.com/runtime/fundamentals/testing/?

Yes! Will do