-
Notifications
You must be signed in to change notification settings - Fork 245
New issue
Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? # to your account
[Doc] Update 22.08 documentation #6216
Conversation
Signed-off-by: Hao Zhu <9665750+viadea@users.noreply.github.com>
Signed-off-by: Hao Zhu <9665750+viadea@users.noreply.github.com>
Co-authored-by: Jason Lowe <jlowe@nvidia.com>
Signed-off-by: Hao Zhu <9665750+viadea@users.noreply.github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Other changes:
- Update the FAQ to note we support Spark 3.2.2
- There is a table formatting issue on gh-pages in https://nvidia.github.io/spark-rapids/docs/compatibility.html#csv-ansi-day-time-interval, possibly due to a lack of a blank line above the text "Currently, the RAPIDS Accelerator only supports the ANSI style." in https://raw.githubusercontent.com/NVIDIA/spark-rapids/branch-22.08/docs/compatibility.md .
- In https://github.com/NVIDIA/spark-rapids/blob/branch-22.08/docs/spark-profiling-tool.md , change
Profiling tool
toProfiling Tool
to be consistent with other page titles at https://nvidia.github.io/spark-rapids/
docs/download.md
Outdated
* Binary read support from parquet | ||
* Apache Iceberg 0.13 support | ||
* Array function support: array_intersect, array_union, array_except and arrays_overlap | ||
* Function from_json support |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
from_json
will be in 22.10 ( PR #6211 )
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed all above.
Can we update https://github.com/NVIDIA/spark-rapids/blob/branch-22.08/docs/configs.md (which is a generated file) to remove |
if a customer setup a databricks cluster, then log in to one cluster node and run some codes through pyspark/spark-shell scripts with rapids-plugin, it will complain that the spark shim loader can not be found, because it is up to |
Co-authored-by: Sameer Raheja <sameerz@users.noreply.github.com>
Co-authored-by: Sameer Raheja <sameerz@users.noreply.github.com>
Co-authored-by: Sameer Raheja <sameerz@users.noreply.github.com>
Signed-off-by: Hao Zhu <9665750+viadea@users.noreply.github.com>
Signed-off-by: Hao Zhu <9665750+viadea@users.noreply.github.com>
Good information. But I feel Databricks users may not really want to SSH to the nodes and run spark-shell. |
Signed-off-by: Hao Zhu <9665750+viadea@users.noreply.github.com>
@sameerz I modified below 1 files to use another parameter Also fixed one typo : spark -> spark-shell For |
…/rapids/RapidsConf.scala Signed-off-by: Hao Zhu <9665750+viadea@users.noreply.github.com>
Signed-off-by: Hao Zhu <9665750+viadea@users.noreply.github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good. I am going to remove the [skip ci]
since there are changes to RapidsConf.scala.
build |
Update the 22.08 documentation.
Note: In this PR, I did not add the download links for 22.08 jars or modify the Tool jar links to avoid link check failure issues.
I will do it right before the final merge PR in another PR.