add support for Lift Template Literal Restriction #23801

Kingwl · 2018-05-01T11:29:31Z

DanielRosenwasser · 2018-05-02T00:02:52Z

src/compiler/scanner.ts

@@ -428,6 +429,16 @@ namespace ts {
          return ch >= CharacterCodes._0 && ch <= CharacterCodes._7;
      }

+      /* @internal */
+      export function isHexDigit(ch: number): boolean {


Neither of these are used outside of scanner.ts, so just don't export them.

DanielRosenwasser · 2018-05-02T00:05:24Z

src/compiler/scanner.ts

@@ -27,7 +27,8 @@ namespace ts {
        getTokenFlags(): TokenFlags;
        reScanGreaterToken(): SyntaxKind;
        reScanSlashToken(): SyntaxKind;
-        reScanTemplateToken(): SyntaxKind;
+        reScanTemplateToken(isTaggedTemplate?: boolean): SyntaxKind;
+        reScanTemplateHead(): SyntaxKind;


@mhegazy, thoughts on just having reScanTemplateToken operate on template heads as well? The implementation could have a Debug.assert like so:

if (token !== SyntaxKind.TemplateHead) { Debug.assert(isTaggedTemplate, "'reScanTemplateToken' should only be called with a template head when in a tagged template."); } else { Debug.assert(token === SyntaxKind.CloseBraceToken, "'reScanTemplateToken' should only be called on a '}' or TemplateHead."); }

DanielRosenwasser

I haven't looked too closely at the logic yet, but the general mechanism between the parser and the scanner looks close to what I would've done, so that's good!

I think what we also need is tests for es5 and es2015. One thing we need to discuss is whether the es2015 target needs the downleveled version.

DanielRosenwasser · 2018-05-02T00:10:14Z

src/compiler/types.ts

@@ -1606,16 +1607,22 @@ namespace ts {
    export interface TemplateHead extends LiteralLikeNode {
        kind: SyntaxKind.TemplateHead;
        parent?: TemplateExpression;
+        /* @internal */
+        notEscapeFlags?: TokenFlags;


Consider just calling these templateFlags.

DanielRosenwasser · 2018-05-02T00:10:34Z

src/compiler/types.ts

@@ -1593,6 +1593,7 @@ namespace ts {
        BinarySpecifier = 1 << 7,   // e.g. `0b0110010000000000`
        OctalSpecifier = 1 << 8,    // e.g. `0o777`
        ContainsSeparator = 1 << 9, // e.g. `0b1100_0101`
+        NotEscape = 1 << 10,        // e.g. `\uhello`


ContainsInvalidEscape

Kingwl · 2018-05-02T01:19:41Z

One thing we need to discuss is whether the es2015 target needs the downleveled version.

In my memory, there are several issues related to escape sequences (es2015) that can be fixed together

Kingwl · 2018-05-02T02:16:01Z

wow, seems ci is upgrade😄

DanielRosenwasser · 2018-05-02T04:11:33Z

src/compiler/parser.ts

@@ -2104,17 +2108,24 @@ namespace ts {
            return allowIdentifierNames ? parseIdentifierName() : parseIdentifier();
        }

-        function parseTemplateExpression(): TemplateExpression {
+        function parseNoSubstitutionTemplate(isTaggedTemplate?: boolean) {


There's no public API, just make this parameter required.

DanielRosenwasser · 2018-05-02T04:46:00Z

tests/baselines/reference/invalidTaggedTemplateEscapeSequences.errors.txt

@@ -0,0 +1,47 @@
+tests/cases/conformance/es2018/invalidTaggedTemplateEscapeSequences.ts(5,18): error TS1125: Hexadecimal digit expected.


We still need to figure out exactly how to prevent the errors from the initial scan...

Kingwl · 2018-05-07T01:53:24Z

should this move to es2018 transform?

DanielRosenwasser · 2018-05-07T04:54:18Z

should this move to es2018 transform?

I would say yes, but only run conditionally. So you only use the ES5-style transformed syntax when an "invalid escape" template string is used, but leave it alone otherwise in the ES2018 transform.

DanielRosenwasser · 2018-05-07T04:55:44Z

Also, I'm not sure if you want to share that logic between each transform, or to just do the ES2015 transform in ES2018. @rbuckton might have a better idea here.

rbuckton · 2018-05-09T22:28:01Z

Also, I'm not sure if you want to share that logic between each transform, or to just do the ES2015 transform in ES2018.

I think the best approach would be to do something similar to what we do for destructuring and pull the logic for template literal down-leveling out of both places. You can then have the esnext.ts transform call this separate API for the tagged templates that need it, and the es2015.ts transform can call it for all tagged templates.

rbuckton · 2018-05-09T22:29:03Z

should this move to es2018 transform?

Not the ES2018 transform, since this is not part of ES2018. Rather, this should go in the esnext.ts transform.

Kingwl · 2018-05-10T16:17:32Z

is that mean i should add a new transformer, eg: taggedTemplate.ts and export the parse function, call the function in es2015 or esnext transformer?

mhegazy · 2018-05-23T23:24:04Z

is that mean i should add a new transformer, eg: taggedTemplate.ts and export the parse function, call the function in es2015 or esnext transformer?

we already have esnext transform. i would put it there..

…m/Kingwl/TypeScript into Lift-Template-Literal-Restriction

Kingwl · 2018-06-06T02:08:22Z

⬆

DanielRosenwasser · 2018-06-08T00:05:08Z

Sorry, I didn't have full context as I was reviewing this. Please ignore the last batch of comments.

Kingwl · 2018-06-08T01:45:27Z

@DanielRosenwasser there are a other commit that i miss
the commit split the transform logic and call them in different transformter
that commit is in my another pc, i'll push it after i finish my work 😢

Kingwl · 2018-06-08T15:19:10Z

ahhh, Sure enough it committed in my another pc

mhegazy · 2018-06-08T17:26:28Z

@rbuckton can you take a look.

RyanCavanaugh

@Kingwl sorry for the delay on this. One last change while you're working on the merge conflicts -- please use the multi-target feature of the test baselines (e.g. @target: es5, es2016) to reduce code duplication. Thanks for the great work here!

Marking as tentative Approve

Kingwl · 2020-01-13T08:31:46Z

@typescript-bot pack this

typescript-bot · 2020-01-13T08:31:49Z

Heya @Kingwl, I've started to run the tarball bundle task on this PR at 569b35e. You can monitor the build here. It should now contribute to this PR's status checks.

sandersn · 2020-02-05T00:15:20Z

One last CI run and I will merge this.

sandersn · 2020-02-05T00:28:38Z

Oh no, it failed on the lint rules we just enabled! I pushed a new commit.

Kingwl · 2020-02-05T13:05:41Z

What is the meaning of For Milestone Bug ... :XD

sandersn · 2020-02-05T17:01:52Z

@Kingwl the PR is intended to fix a bug that has been accepted into a milestone. That contrasts with "For Backlog Bug", which are PRs intended to fix a backlog bug. This PR technically fixes a backlog bug, but it's an Ecmascript conformance bug, so I think it should have been put in a milestone long ago.

Kingwl · 2020-02-06T14:48:05Z

@typescript-bot pack this.

typescript-bot · 2020-02-06T14:48:13Z

Heya @Kingwl, I've started to run the tarball bundle task on this PR at f8b9bbb. You can monitor the build here. It should now contribute to this PR's status checks.

This problem was introduced in 70399e1 (from PR microsoft#23801), which added a `visitTaggedTemplateExpression` case for `TaggedTemplateExpression`, before that, it would fallback to the default of `visitNode`. So re-add that happen in `processTaggedTemplateExpression`. Since it doesn't hurt, I left a `Debug.checkDefined(property.name)` instead of `!`-ing it. Fixes microsoft#38558.

This problem was introduced in 70399e1 (from PR #23801), which added a `visitTaggedTemplateExpression` case for `TaggedTemplateExpression`, before that, it would fallback to the default of `visitNode`. So re-add that happen in `processTaggedTemplateExpression`. Since it doesn't hurt, I left a `Debug.checkDefined(property.name)` instead of `!`-ing it. Fixes #38558.

Kingwl added 2 commits May 1, 2018 19:24

add support for Lift Template Literal Restriction

344d505

rename file and improve comment and tests

6ef797f

DanielRosenwasser reviewed May 2, 2018

View reviewed changes

DanielRosenwasser requested changes May 2, 2018

View reviewed changes

fix NoSubstitutionTemplateLiteral support

3932d20

DanielRosenwasser requested changes May 2, 2018

View reviewed changes

Kingwl added 3 commits May 24, 2018 23:04

Merge branch 'master' into Lift-Template-Literal-Restriction

dbbcdae

Merge branch 'Lift-Template-Literal-Restriction' of https://github.co…

c642abe

…m/Kingwl/TypeScript into Lift-Template-Literal-Restriction

extract tagged template and add more test

e92b6bc

Kingwl force-pushed the Lift-Template-Literal-Restriction branch from e92b6bc to 2f49205 Compare May 30, 2018 09:50

Kingwl mentioned this pull request Jun 6, 2018

Support for --target ES2018 #20463

Closed

3 tasks

Kingwl force-pushed the Lift-Template-Literal-Restriction branch from 2f49205 to e92b6bc Compare June 8, 2018 15:00

Kingwl added 2 commits June 8, 2018 23:03

Merge branch 'master' into Lift-Template-Literal-Restriction

9bfb15b

avoid useless parameter

2c973b1

Kingwl added 2 commits June 9, 2018 02:47

fix incorrect return node if cannot transform

c2580ed

accept baseline

21c91d6

Kingwl added 2 commits August 27, 2019 17:06

fix again

36c3bb3

fix again

b7f5c77

RyanCavanaugh approved these changes Jan 8, 2020

View reviewed changes

Kingwl added 2 commits January 10, 2020 00:09

Merge branch 'master' into Lift-Template-Literal-Restriction

fd47863

use multiple target

569b35e

sandersn added For Milestone Bug PRs that fix a bug with a specific milestone and removed For Milestone Bug PRs that fix a bug with a specific milestone labels Jan 31, 2020

Merge branch 'master' into Lift-Template-Literal-Restriction

45406d4

fix space lint

f8b9bbb

sandersn merged commit 70399e1 into microsoft:master Feb 5, 2020

Kingwl deleted the Lift-Template-Literal-Restriction branch February 6, 2020 14:47

rbuckton mentioned this pull request May 8, 2020

Downlevel emit for template strings should use 'void 0' not 'undefined' #38430

Closed

elibarzilay mentioned this pull request May 15, 2020

Make processTaggedTemplateExpression visit a returned node #38596

Merged

Kingwl mentioned this pull request Jul 27, 2020

raw tagged string doesn't support for \x #39715

Closed

Kingwl mentioned this pull request Feb 20, 2021

\u in String.raw #42887

Closed

graphemecluster mentioned this pull request Dec 9, 2022

fix(396): (Dis)allow Octal and Bad Escape Sequences in String and (Tagged) Template Literals #51837

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add support for Lift Template Literal Restriction #23801

add support for Lift Template Literal Restriction #23801

Kingwl commented May 1, 2018

DanielRosenwasser May 2, 2018

DanielRosenwasser May 2, 2018 •

edited

Loading

DanielRosenwasser left a comment

DanielRosenwasser May 2, 2018

DanielRosenwasser May 2, 2018

Kingwl commented May 2, 2018

Kingwl commented May 2, 2018

DanielRosenwasser May 2, 2018

DanielRosenwasser May 2, 2018

Kingwl commented May 7, 2018

DanielRosenwasser commented May 7, 2018 •

edited

Loading

DanielRosenwasser commented May 7, 2018

rbuckton commented May 9, 2018

rbuckton commented May 9, 2018

Kingwl commented May 10, 2018

mhegazy commented May 23, 2018

Kingwl commented Jun 6, 2018

DanielRosenwasser commented Jun 8, 2018

Kingwl commented Jun 8, 2018

Kingwl commented Jun 8, 2018

mhegazy commented Jun 8, 2018

RyanCavanaugh left a comment

Kingwl commented Jan 13, 2020

typescript-bot commented Jan 13, 2020 •

edited

Loading

sandersn commented Feb 5, 2020

sandersn commented Feb 5, 2020

Kingwl commented Feb 5, 2020

sandersn commented Feb 5, 2020

Kingwl commented Feb 6, 2020

typescript-bot commented Feb 6, 2020 •

edited

Loading

		@@ -0,0 +1,47 @@
		tests/cases/conformance/es2018/invalidTaggedTemplateEscapeSequences.ts(5,18): error TS1125: Hexadecimal digit expected.

add support for Lift Template Literal Restriction #23801

add support for Lift Template Literal Restriction #23801

Conversation

Kingwl commented May 1, 2018

DanielRosenwasser May 2, 2018

Choose a reason for hiding this comment

DanielRosenwasser May 2, 2018 • edited Loading

Choose a reason for hiding this comment

DanielRosenwasser left a comment

Choose a reason for hiding this comment

DanielRosenwasser May 2, 2018

Choose a reason for hiding this comment

DanielRosenwasser May 2, 2018

Choose a reason for hiding this comment

Kingwl commented May 2, 2018

Kingwl commented May 2, 2018

DanielRosenwasser May 2, 2018

Choose a reason for hiding this comment

DanielRosenwasser May 2, 2018

Choose a reason for hiding this comment

Kingwl commented May 7, 2018

DanielRosenwasser commented May 7, 2018 • edited Loading

DanielRosenwasser commented May 7, 2018

rbuckton commented May 9, 2018

rbuckton commented May 9, 2018

Kingwl commented May 10, 2018

mhegazy commented May 23, 2018

Kingwl commented Jun 6, 2018

DanielRosenwasser commented Jun 8, 2018

Kingwl commented Jun 8, 2018

Kingwl commented Jun 8, 2018

mhegazy commented Jun 8, 2018

RyanCavanaugh left a comment

Choose a reason for hiding this comment

Kingwl commented Jan 13, 2020

typescript-bot commented Jan 13, 2020 • edited Loading

sandersn commented Feb 5, 2020

sandersn commented Feb 5, 2020

Kingwl commented Feb 5, 2020

sandersn commented Feb 5, 2020

Kingwl commented Feb 6, 2020

typescript-bot commented Feb 6, 2020 • edited Loading

DanielRosenwasser May 2, 2018 •

edited

Loading

DanielRosenwasser commented May 7, 2018 •

edited

Loading

typescript-bot commented Jan 13, 2020 •

edited

Loading

typescript-bot commented Feb 6, 2020 •

edited

Loading