Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Not allowing Infinite values for Double #1769

Open
FreCap opened this issue Oct 28, 2022 · 0 comments
Open

Not allowing Infinite values for Double #1769

FreCap opened this issue Oct 28, 2022 · 0 comments

Comments

@FreCap
Copy link

FreCap commented Oct 28, 2022

Problem you have encountered:

Error sending insert request, table: xxxx
com.google.cloud.bigquery.BigQueryException: java.lang.IllegalArgumentException
at com.google.cloud.bigquery.BigQueryException.translateAndThrow(BigQueryException.java:100)
at com.google.cloud.bigquery.BigQueryImpl.insertAll(BigQueryImpl.java:979)
...
Caused by: java.lang.IllegalArgumentException
    at com.google.common.base.Preconditions.checkArgument(Preconditions.java:128)
    at com.google.api.client.util.Preconditions.checkArgument(Preconditions.java:35)    
    at com.google.api.client.json.JsonGenerator.serialize(JsonGenerator.java:134)
    at com.google.api.client.json.JsonGenerator.serialize(JsonGenerator.java:173)
    at com.google.api.client.json.JsonGenerator.serialize(JsonGenerator.java:173)
    at com.google.api.client.json.JsonGenerator.serialize(JsonGenerator.java:173)
    at com.google.api.client.json.JsonGenerator.serialize(JsonGenerator.java:146)
    at com.google.api.client.json.JsonGenerator.serialize(JsonGenerator.java:173)
    at com.google.api.client.json.JsonGenerator.serialize(JsonGenerator.java:105)

It seems that the json serializer is very restrictive (not able to change/configure it) in handling Infinity values, which are actually supported by BigQuery and Double standard.

https://github.com/googleapis/google-http-java-client/blob/main/google-http-client/src/main/java/com/google/api/client/json/JsonGenerator.java#L134

What you expected to happen:

I would expect the JsonGenerator, to be able to pass Infinity (as a string?) and not hard fail.

e.g. draft

        double doubleValue = ((Number) value).doubleValue();
        if (Double.isInfinite(doubleValue)){ // or   if (Double.isInfinite(doubleValue && allowDoubleInfinityAsString))
          writeString("infinity");
        } else {
          Preconditions.checkArgument(!Double.isInfinite(doubleValue) && !Double.isNaN(doubleValue));
          writeNumber(doubleValue);
        }

The alternative of cleaning the data with Infinity values before inserting it would imply loss of information due to the transportation mechanism, which shouldn't be a limitation itself

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant