GpuAdd supports ANSI mode. #3537
Conversation
And let GPU Multiply fall back to CPU when ansi mode is enabled. Signed-off-by: Firestarman <firestarmanllc@gmail.com>
|
build |
| ShortGen(min_val = 1, max_val = 100, special_cases=[]), | ||
| IntegerGen(min_val = 1, max_val = 1000, special_cases=[]), | ||
| LongGen(min_val = 1, max_val = 3000, special_cases=[]), | ||
| float_gen, double_gen, |
There was a problem hiding this comment.
nit: I think we can have a follow on issue to re-enable float, double, and decimal processing in ANSI mode. Each of them handle overflows already in different ways that should be okay.
| @@ -1380,7 +1380,7 @@ object GpuOverrides extends Logging { | |||
| ExprChecks.mathUnary, | |||
| (a, conf, p, r) => new UnaryExprMeta[Log1p](a, conf, p, r) { | |||
| override def convertToGpu(child: Expression): GpuExpression = | |||
| GpuLog(GpuAdd(child, GpuLiteral(1d, DataTypes.DoubleType))) | |||
| GpuLog(GpuAdd(child, GpuLiteral(1d, DataTypes.DoubleType), SQLConf.get.ansiEnabled)) | |||
There was a problem hiding this comment.
nit: This is a little misleading, because there is no fail on error overflow processing on doubles, and doubles are the only thing supported by Log1p
|
|
||
| override def tagSelfForAst(): Unit = { | ||
| super.tagSelfForAst() | ||
| if (ansiEnabled) { |
There was a problem hiding this comment.
nit: It would be good to check the data type, because double and float don't have special overflow processing and should be able to work in AST mode.
| override def tagSelfForAst(): Unit = { | ||
| super.tagSelfForAst() | ||
| if (ansiEnabled) { | ||
| willNotWorkInAst("AST Addition does not support ansi mode.") |
There was a problem hiding this comment.
| willNotWorkInAst("AST Addition does not support ansi mode.") | |
| willNotWorkInAst("AST Addition does not support ANSI mode.") |
| @@ -62,6 +63,26 @@ def test_multiplication(data_gen): | |||
| f.col('a') * f.col('b')), | |||
| conf=allow_negative_scale_of_decimal_conf) | |||
|
|
|||
| # No overflow gens here because we just focus on verifying the fallback to CPU when | |||
| # enabling ansi mode. But overflows will fail the tests because CPU runs raise | |||
There was a problem hiding this comment.
| # enabling ansi mode. But overflows will fail the tests because CPU runs raise | |
| # enabling ANSI mode. But overflows will fail the tests because CPU runs raise |
| @@ -1709,6 +1718,10 @@ object GpuOverrides extends Logging { | |||
| } | |||
| case _ => // NOOP | |||
| } | |||
|
|
|||
| if (SQLConf.get.ansiEnabled) { | |||
| willNotWorkOnGpu("GPU Multiplication does not support ansi mode") | |||
There was a problem hiding this comment.
| willNotWorkOnGpu("GPU Multiplication does not support ansi mode") | |
| willNotWorkOnGpu("GPU Multiplication does not support ANSI mode") |
There was a problem hiding this comment.
I'll do this in a follow on issue as @firestarman is now out on vacation.
And let Multiply fall back to CPU when ANSI mode is enabled.
This closes #3472
Signed-off-by: Firestarman firestarmanllc@gmail.com