Bug: Token Regex for fileuniqueid* matches too broadly

[heinrich-ulbricht]

This Regex matches more than just fileuniqueid* tokens:

pnpframework/src/lib/PnP.Framework/Provisioning/ObjectHandlers/Utilities/ListItemUtilities.cs

Line 283 in 83eaf1f

Regex fileUniqueIdToken = new Regex("(?<token>[{]{1,2}(?:fileuniqueid:fileuniqueidencoded:)|[^}]*[}]{1,2})(?:[^{]*)", RegexOptions.IgnoreCase | RegexOptions.Multiline | RegexOptions.Compiled);

This will be handled OK since exceptions are swallowed, but calls are made that are not necessary (for every field content containing curly braces). So this Regex could be improved to not trigger those redundant calls.

Alternative easy improvement: after this line:

pnpframework/src/lib/PnP.Framework/Provisioning/ObjectHandlers/Utilities/ListItemUtilities.cs

Line 308 in 83eaf1f

var tokenParts = replacementVal.Trim(new char[] { '{', '}' }).Split(':');

Add:

if (tokenParts.Length < 2)
{
    continue;
}

This catches most mismatches and prevents further redundant calls.

[heinrich-ulbricht]

Ramping this up to bug as I got a case where some content was matched by the RegEx and tokenParts[1] was an empty string. This causes the ID of a folder to be loaded here:

pnpframework/src/lib/PnP.Framework/Provisioning/ObjectHandlers/Utilities/ListItemUtilities.cs

Lines 323 to 325 in ab5fa53

	var folder = web.GetFolderByServerRelativePath(ResourcePath.FromDecodedUrl(tokenParts[1]));
	folder.EnsureProperties(f => f.UniqueId);
	UniqueId = folder.UniqueId;

But the invalid tokenParts then throw here with invalid pattern exception:

pnpframework/src/lib/PnP.Framework/Provisioning/ObjectHandlers/Utilities/ListItemUtilities.cs

Line 340 in ab5fa53

value = Regex.Replace(value, $"{{{tokenParts[0]}:{tokenParts[1]}}}", replTest);

Proposed fix:

WikiTransformationProject@277dd27

[carlosmiguelsns]

I noticed that there is a RegEx responsible for capturing tokens in the UpdateListItem process. However, if the list item contains a JSON, the existing RegEx does not validate correctly.

There could probably be a rewrite of the regex here or in the next process of the condition to exclude tokens that start with a quotation mark or single quote, because it identifies a JSON property.

However, I've made an improvement to the RegEx:

Regex fileUniqueIdToken = new Regex("(?<token>\\{\\{(?!\\\")[^{}]+?:[^{}]+?\\}\\}|(?<!\\{)\\{(?!\\\")[^{}]+?:[^{}]+?\\}(?!\\}))", RegexOptions.IgnoreCase | RegexOptions.Multiline | RegexOptions.Compiled);

Matching - all tokens that containst at list :

• {fileuniqueid:[siteRelativePath]}
• {fileuniqueidencoded:[siteRelativePath]}
• {othertokenname:anyvalue} or {{tokenname:value}}

I'm not specifying fileuniqueid or fileuniqueidencoded because it's implicit in the RegEx.

Can someone please validate if there is a match that is not being adressed?

If this is the right solution, I can do a PR of my code