Improve docs for TableProvider::supports_filters_pushdown and remove deprecated function
#9923
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
logical-expr
alamb
commented
Apr 3, 2024
| @@ -1430,12 +1430,12 @@ impl TableProvider for DataFrameTableProvider { | |||
| Some(&self.plan) | |||
| } | |||
|
|
|||
| fn supports_filter_pushdown( | |||
| fn supports_filters_pushdown( | |||
There was a problem hiding this comment.
Switches to using the non deprecated but very similarly named API
| /// to optimise data retrieval. | ||
| /// Note: the returned vector much have the same size as the filters argument. | ||
| #[allow(deprecated)] | ||
| /// Specify if DataFusion should provide filter expressions to the |
There was a problem hiding this comment.
I tried to improve the docs about what this was doing
| @@ -154,28 +158,31 @@ pub trait TableProvider: Sync + Send { | |||
| limit: Option<usize>, | |||
| ) -> Result<Arc<dyn ExecutionPlan>>; | |||
|
|
|||
| /// Tests whether the table provider can make use of a filter expression | |||
| /// to optimise data retrieval. | |||
| #[deprecated(since = "20.0.0", note = "use supports_filters_pushdown instead")] | |||
There was a problem hiding this comment.
This has been deprecated for quite a while
|
I was similarly confused when I first ran across this before figuring it out. Thanks for fixing. |
phillipleblanc
approved these changes
Apr 3, 2024
Co-authored-by: Phillip LeBlanc <phillip@leblanc.tech>
github-actions
bot
removed
the
documentation
alamb
commented
Apr 3, 2024
There was a problem hiding this comment.
Thank you for the review @phillipleblanc
cc @avantgardnerio as I think you maybe added the original API way back
|
Thanks again @avantgardnerio and @phillipleblanc |
Michael-J-Ward
added a commit
to Michael-J-Ward/datafusion-python
that referenced
this pull request
May 13, 2024
…ters_pushdown Deprecated function removed in apache/datafusion#9923
andygrove
pushed a commit
to apache/datafusion-python
that referenced
this pull request
May 14, 2024
* chore: upgrade datafusion Deps Ref #690 * update concat and concat_ws to use datafusion_functions Moved in apache/datafusion#10089 * feat: upgrade functions.rs Upstream is continuing it's migration to UDFs. Ref apache/datafusion#10098 Ref apache/datafusion#10372 * fix ScalarUDF import * feat: remove deprecated suppors_filter_pushdown and impl supports_filters_pushdown Deprecated function removed in apache/datafusion#9923 * use `unnest_columns_with_options` instead of deprecated `unnest_column_with_option` * remove ScalarFunction wrappers These relied on upstream BuiltinScalarFunction, which are now removed. Ref apache/datafusion#10098 * update dataframe `test_describe` `null_count` was fixed upstream. Ref apache/datafusion#10260 * remove PyDFField and related methods DFField was removed upstream. Ref: apache/datafusion#9595 * bump `datafusion-python` package version to 38.0.0 * re-implement `PyExpr::column_name` The previous implementation relied on `DFField` which was removed upstream. Ref: apache/datafusion#9595
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes #9922
part of #7013
Rationale for this change
While debugging something with @pauldix yesterday, we were somewhat confused with the old API (as the default impl calls the deprecated function) and the docs weren't clear
We figured it out, but it was a paper cut that caused more confusion that it should have
What changes are included in this PR?
TableProvider::supports_filters_pushdownTableProviderFilterPushDownTableProvider::supports_filter_pushdownAre these changes tested?
Yes by CI
Are there any user-facing changes?
Docs, and the deprecated API is now removed