From b0bfe1c356f0b33e1fb83ac0c0cb47ba74d13444 Mon Sep 17 00:00:00 2001 From: Oleg Bespalov Date: Thu, 29 Aug 2024 15:58:20 +0200 Subject: [PATCH 1/3] Document maintanance of tc39 --- js/tc39/README.md | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/js/tc39/README.md b/js/tc39/README.md index 5d317841dfb..5fa49530a84 100644 --- a/js/tc39/README.md +++ b/js/tc39/README.md @@ -7,15 +7,23 @@ Ways to use it: that was tested with this module 2. Run `go test &> out.log` -The full list of failing tests, and the error, is in `breaking_test_errors-*.json`. All errors list there with the corresponding error will *not* be counted as errors - this is what the test expects, those specific errors. -Due to changes to Sobek it's common for the error to change, or there to be now a new error on the previously passing test, or (hopefully) a test that was not passing but now is. -In all of those cases `breaking_test_errors-*.json` needs to be updated. Run the test with `-update` flag to update: `go test -update` - -NOTE: some text editors/IDEs will try to parse files ending in `json` as JSON, which given the size of `breaking_test_errors-*.json` might be a problem when it's not actually a JSON (before the edit). So it might be a better idea to name it something different if editing by hand and fix it later. +The full list of failing tests, and the error, is in `breaking_test_errors-*.json`. All errors list there with the corresponding error will *not* be counted as errors - this is what the test expects, those specific errors. See reasons for this at the end of this document. This is a modified version of [the code in the original goja repo](https://github.com/dop251/goja/blob/master/tc39_test.go) that Sobek was forked from. +## Maintaining the tests + +There are few things to keep in mind when maintaining the tests. + +* For the most of the version we aim to have the same version of the test suite as the one that [uses Sobek](https://github.com/grafana/sobek/blob/main/.tc39_test262_checkout.sh#L3). So if the Sobek brings a new version of the test suite, we should update the version of the test suite in this package as well. +* The new version of Sobek could bring a new functionality, so you might need to re-evaluate the `featuresBlockList`, `skipList` and any other list in the `tc39_test.go` file. The way to go is also keeping this lists closer to the Sobek, however they are not the same, so you might need to adjust them. +* Due to changes to Sobek it's common for the error to change, or there to be now a new error on the previously passing test, or (hopefully) a test that was not passing but now is. +In all of those cases `breaking_test_errors-*.json` needs to be updated. Run the test with `-update` flag to update: `go test -update`. +* Important to mention that there should be a balance between just updating the list of the errors and actually fixing errors. So it's recommended to check the diff of the `breaking_test_errors-*.json` and case by case decide if the error should be updated or the test should be fixed. + +> [!NOTE] +> some text editors/IDEs will try to parse files ending in `json` as JSON, which given the size of `breaking_test_errors-*.json` might be a problem when it's not actually a JSON (before the edit). So it might be a better idea to name it something different if editing by hand and fix it later. ## Reasons for recording breaking_test_errors.json From 35db3a4ca674397c82317c0e5629d3bd03ae84c9 Mon Sep 17 00:00:00 2001 From: Oleg Bespalov Date: Thu, 29 Aug 2024 16:37:16 +0200 Subject: [PATCH 2/3] Apply suggestions from code review Co-authored-by: Mihail Stoykov <312246+mstoykov@users.noreply.github.com> --- js/tc39/README.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/js/tc39/README.md b/js/tc39/README.md index 5fa49530a84..7969c5e0b0d 100644 --- a/js/tc39/README.md +++ b/js/tc39/README.md @@ -22,8 +22,6 @@ There are few things to keep in mind when maintaining the tests. In all of those cases `breaking_test_errors-*.json` needs to be updated. Run the test with `-update` flag to update: `go test -update`. * Important to mention that there should be a balance between just updating the list of the errors and actually fixing errors. So it's recommended to check the diff of the `breaking_test_errors-*.json` and case by case decide if the error should be updated or the test should be fixed. -> [!NOTE] -> some text editors/IDEs will try to parse files ending in `json` as JSON, which given the size of `breaking_test_errors-*.json` might be a problem when it's not actually a JSON (before the edit). So it might be a better idea to name it something different if editing by hand and fix it later. ## Reasons for recording breaking_test_errors.json From f0850b2a7de866dd1aa3b8a378720e653b043781 Mon Sep 17 00:00:00 2001 From: Oleg Bespalov Date: Fri, 30 Aug 2024 10:03:56 +0200 Subject: [PATCH 3/3] Apply suggestions from code review --- js/tc39/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/js/tc39/README.md b/js/tc39/README.md index 7969c5e0b0d..9919d3951ef 100644 --- a/js/tc39/README.md +++ b/js/tc39/README.md @@ -16,7 +16,7 @@ repo](https://github.com/dop251/goja/blob/master/tc39_test.go) that Sobek was fo There are few things to keep in mind when maintaining the tests. -* For the most of the version we aim to have the same version of the test suite as the one that [uses Sobek](https://github.com/grafana/sobek/blob/main/.tc39_test262_checkout.sh#L3). So if the Sobek brings a new version of the test suite, we should update the version of the test suite in this package as well. +* For most of the time, we aim to have the same version (commit hash) of the test suite as the one that [uses Sobek](https://github.com/grafana/sobek/blob/main/.tc39_test262_checkout.sh#L3). So if Sobek brings a new version of the test suite, we should update the version of the test suite in this package as well. * The new version of Sobek could bring a new functionality, so you might need to re-evaluate the `featuresBlockList`, `skipList` and any other list in the `tc39_test.go` file. The way to go is also keeping this lists closer to the Sobek, however they are not the same, so you might need to adjust them. * Due to changes to Sobek it's common for the error to change, or there to be now a new error on the previously passing test, or (hopefully) a test that was not passing but now is. In all of those cases `breaking_test_errors-*.json` needs to be updated. Run the test with `-update` flag to update: `go test -update`.