Skip to content

Fix for 15256 - bad generated SQL for DefaultValue with line breaks. #20304

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Merged
merged 1 commit into from
Mar 26, 2020

Conversation

lajones
Copy link
Contributor

@lajones lajones commented Mar 17, 2020

Fix for #15256 - bad SQL was generated for DefaultValue with line breaks.

@lajones lajones requested a review from bricelam March 17, 2020 01:47
@lajones lajones self-assigned this Mar 17, 2020
@AndriySvyryd
Copy link
Member

Also apply the fix to ComputedColumnSql in SqlServerMigrationsSqlGenerator

@lajones
Copy link
Contributor Author

lajones commented Mar 20, 2020

@AndriySvyryd My mistake - I shouldn't have been applying it to DefaultValueSql in the first place - only to DefaultValue. For both DefaultValueSql and ComputedColumnSql it is assumed the string you are passing in is already in the form of valid SQL - and we shouldn't be trying to change it. But for DefaultValue if you pass in a string we should escape line breaks as otherwise we'll end up with invalid SQL.

@smitpatel
Copy link
Contributor

For both DefaultValueSql and ComputedColumnSql it is assumed the string you are passing in is already in the form of valid SQL

I wonder if that assumption actually says anything about us trying to indent the SQL. Users may be totally unaware of. Even if a valid SQL passed in, our indented new line can break it.

@lajones lajones force-pushed the 20200313_Issue15256_02 branch from b8cfcde to e02787e Compare March 21, 2020 02:12
@lajones lajones force-pushed the 20200313_Issue15256_02 branch from e02787e to 285027f Compare March 21, 2020 02:15
@lajones
Copy link
Contributor Author

lajones commented Mar 21, 2020

@smitpatel I think the remark about indentation was a red-herring. The problem really was we would generate something like this:

CREATE TABLE [TestEntity] (
    [Id] int NOT NULL IDENTITY,
    [Name] nvarchar(max) NULL DEFAULT (Do not
do
this
)

And that's invalid SQL - so users of that would definitely notice. Now we'll generate something like this:

CREATE TABLE [TestEntity] (
    [Id] int NOT NULL IDENTITY,
    [Name] nvarchar(max) NULL DEFAULT CONCAT(N'Do not', CHAR (13), CHAR(10), N'do', CHAR (13), CHAR(10), N'this')
)

which is valid.

@AndriySvyryd
Copy link
Member

@lajones Ok, but what do DefaultValueSql and ComputedColumnSql with line breaks produce when we script migrations?

@lajones lajones changed the title Fix for 15256 - bad generated SQL for DefaultValueSql with line breaks. Fix for 15256 - bad generated SQL for DefaultValue with line breaks. Mar 21, 2020
@lajones
Copy link
Contributor Author

lajones commented Mar 21, 2020

@AndriySvyryd - they would produce bad SQL but that's the user's responsibility. The difference is that for DefaultValue() the user is assumed to be passing in a value, and EF is responsible for converting that value to valid SQL, whereas for DefaultValueSql() and ComputedColumnSql() the user is assumed to be passing in SQL and they are responsible for making sure that the SQL is valid.

There are an infinite number of ways that a user could pass invalid SQL to DefaultValueSql() and ComputedColumnSql() and us trying to guess and "fix" it will probably end up preventing someone else's scenario.

For instance, we cannot put single quotes around everything (like we do for strings passed to DefaultValue()) because it's perfectly possible that the user wants to call a function and end up with e.g. DEFAULT someFunction() which is very different from DEFAULT 'someFunction()'. Instead, if the user wants single quotes, the user is responsible for putting them in the string which they pass to DefaultValueSql() or ComputedColumnSql() - so we cannot be sure that replacing line breaks with CONCAT('abc', CHAR(13), CHAR(10), 'xyz') is the right thing to do as we're not sure that abc and xyz were meant to be strings to start with.

@smitpatel
Copy link
Contributor

SELECT TOP(1) a.Column
From table as a
Where a.text = N'abc
   def
   ghi
end';

A valid SQL. If you indent it, gives you incorrect results.

@lajones
Copy link
Contributor Author

lajones commented Mar 21, 2020

OK. Agreed. This PR fixes the problem for DefaultValue - because the value will now always be on the same line as the DEFAULT statement. So please review that.

But I agree it does not address the issue for DefaultValueSql or ComputedColumnSql using SQL such as @smitpatel wrote above. We cannot solve that issue by the same method - for the reasons I stated above. So that will have to be a separate PR (presumably involving a different mode for IndentedStringBuilder).

@lajones
Copy link
Contributor Author

lajones commented Mar 21, 2020

Also note - a user of DefaultValueSql() or ComputedColumnSql() could solve the indentation problem right now. They could pass in the following:

SELECT TOP(1) a.Column
From table as a
Where a.text = CONCAT(N'abc', CHAR(13), CHAR(10), N'   def', CHAR(13), CHAR(10), N'   ghi', CHAR(13), CHAR(10), N'end');

which also makes clear their expectations of how newlines are stored in the database.

But a user of DefaultValue cannot solve the same problem - because EF is in charge of formatting the SQL. That's what this PR addresses.

@bricelam
Copy link
Contributor

bricelam commented Mar 23, 2020

@smitpatel This is only an issue with idempotent scripts, so I'm not really worried about the migrationBuilder.Sql() cases since the workaround is just to write your SQL the same way we do.

I only feel obligated to fix the cases where we generate bad SQL for a literal value that the user correctly provided. Because in these cases, there isn't a good workaround.

@smitpatel
Copy link
Contributor

@bricelam - I am not referring to migrationBuilder.Sql() API. I am referring to HasDefaultValueSql API. I don't have strong preference over trying to fix it or not. It would be just hard to detect errors.

@AndriySvyryd
Copy link
Member

At least file an issue to track this

@lajones
Copy link
Contributor Author

lajones commented Mar 23, 2020

@bricelam @AndriySvyryd @smitpatel I'm happy to file the new issue to look into DefaultValueSql or ComputedColumnSql, but can we sign off on this please?

@lajones
Copy link
Contributor Author

lajones commented Mar 24, 2020

@AndriySvyryd I filed #20391 to track that issue.

@ajcvickers
Copy link
Contributor

@smitpatel @AndriySvyryd @bricelam Given that we have an issue tracking SQL cases, is there anything left to do here? If not, can someone sign off?

@lajones lajones merged commit e6507f6 into master Mar 26, 2020
@lajones lajones deleted the 20200313_Issue15256_02 branch March 26, 2020 21:43
@lajones
Copy link
Contributor Author

lajones commented Mar 26, 2020

Many thanks to @lauxjpn and in particular his update to the MySql provider for inspiration on this.

ajcvickers added a commit that referenced this pull request Mar 28, 2020
Fix two issues building and testing on Linux introduced by #20304 (bad generated SQL for DefaultValue with line breaks /cc @lajones)
* Test literals were using machine line endings
* Replacement mechanism for new lines in literals did not handle a single `\n`

Fixes #20439
Fixes #20438
ajcvickers added a commit that referenced this pull request Mar 28, 2020
Fix two issues building and testing on Linux introduced by #20304 (bad generated SQL for DefaultValue with line breaks /cc @lajones)
* Test literals were using machine line endings
* Replacement mechanism for new lines in literals did not handle a single `\n`

Fixes #20439
Fixes #20438
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants