WIP: design doc for windows-native test-runner

docs/2018-07-18-windows-native-test-runner.md

@@ -0,0 +1,96 @@
+## How `tools/test/test-setup.sh` works
+
+1.  Absolutizes path-storing environment variables.
+
+    **Why**: When Bazel creates the action and its list of environment
+    variables, Bazel does not yet know under which absolute path (on what
+    machine) the action will run. The `test-setup.sh` script absolutizes the
+    envvars by making them relative to `$PWD`.
+
+1.  Sets `$USER` using `whoami`.
+
+1.  Creates some directories, e.g. for the undeclared outputs, the shard status
+    file, the XML test log, the test temp directory.
+
+1.  Exports some environment variables.
+
+    **Why**: Tests and test runners need envvars such as `$TEST_TMPDIR` or
+    `$TEST_SHARD_INDEX`.
+
+1.  Defines `rlocation`.
+
+    **Why**:
+
+    - So `test-setup.sh` can look up the test executable's path.
+
+    - For the benefit of shell tests, should this be a shell test. Tests that
+      use the Bash runfiles library (in `@bazel_tools//tools/bash/runfiles`) do
+      not need this `rlocation` function.
+
+1.  Defines `encode_output_file` that:
+
+    -   Runs `perl` and `sed` to sanitize the test's output so it's safe to
+        embed in the test XML log's CDATA section. `write_xml_output_file` calls
+        this function.
+
+1.  Defines `write_xml_output_file` that:
+
+    - Creates `$XML_OUTPUT_FILE` from the test log, using `encode_output_file`
+
+    - Removes `${XML_OUTPUT_FILE}.log`
+
+      **Why**: `${XML_OUTPUT_FILE}.log` is a temporary file containing the
+      test's output.
+
+1.  Changes the current directory (if `$COVERAGE_DIR` is empty).
+
+    **Why**: `TEST_SRCDIR` is the runfiles directory. Changing into it prevents
+    the test from running in the execroot, where other actions normally run.
+    Keep in mind that on Windows th runfiles directory is empty except for the
+    runfiles manifest so tests always need to use the runfiles libraries (in
+    `@bazel_tools//`) if they need to look up data-dependencies.
+
+1.  Adds `.` to `$PATH`.
+
+    **Why**: Supposedly so it can run the test executable if it's directly in
+    the current directory.
+
+    This step is no longer necessary, because the test executable's path is
+    always absolute.
+
+1.  Sets `$TEST_PATH` to the absolute path of the test executable.
+
+    If `$TEST_SHORT_EXEC_PATH` is defined, it sets an alternative `$TEST_PATH`.
+
+    **Why**: to avoid too long paths on Windows with remote execution.
+
+1.  Traps all signals to be handled by `write_xml_output_file`.
+
+1.  Runs the test:
+
+    If required tools are available, runs the test as a subprocess and redirects
+    its output to `${XML_OUTPUT_FILE}.log` while also streaming it to stdout via
+    `less`; otherwise runs the test directly and `tee` its output into
+    `${XML_OUTPUT_FILE}.log`.
+
+    In both cases, runs the test via the `tools/test/collect_coverage.sh` if
+    requested, which:
+
+    1.  Runs in "legacy mode" if `$LCOV_MERGER` is undefined.
+
+        This mode triggers Google-specific code paths that rely on
+        `/usr/bin/lcov`. We don't need to support this mode on Windows.
+
+    1.  Absolutizes and exports some path-storing envvars (e.g.
+        `$COVERAGE_MANIFEST`, `$COVERAGE_DIR`).
+
+    1.  Changes the current directory to the test's workspace, runs the
+        test, stores the exit code.
+
+    1.  `exec()`s the `$LCOV_MERGER`
+
+1.  Resets all signal handlers, calls `write_xml_output_file`.
+
+1.  Writes the manifest file and annotation file for undeclared outputs.
+
+1.  Creates a `zip` file of the undeclared outputs.