Historical feature retrieval e2e test #1067
Conversation
khorshuheng
requested review from
davidheryanto,
pyalex,
woop and
zhilingc
as code owners
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: khorshuheng The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
| Feature("daily_transactions", ValueType.DOUBLE), | ||
| Feature("total_transactions", ValueType.DOUBLE), | ||
| ], | ||
| batch_source=FileSource( |
There was a problem hiding this comment.
I think we need to change the file source to have named arguments and not positional, even if they are required. The only positional argument should be the path, and it should come first. Otherwise its hard to understand what is being defined here.
| creation_date = retrieval_date - timedelta(1) | ||
|
|
||
| customers = [1001, 1002, 1003, 1004, 1005] | ||
| daily_transactions = [np.random.rand() * 10 for _ in customers] |
|
|
||
| expected_joined_df = pd.DataFrame( | ||
| { | ||
| "event_timestamp": [retrieval_date for _ in customers] |
There was a problem hiding this comment.
Are you at all testing the as_of join functionality? Looks like you just have one timestamp in the range?
Signed-off-by: Khor Shu Heng <khor.heng@gojek.com>
khorshuheng
force-pushed
the
historical-retrieval-e2e-test
branch
from
0f7dc51 to
342c49d
Compare
khorshuheng
changed the title
Signed-off-by: Khor Shu Heng <khor.heng@gojek.com>
|
@khorshuheng: The following tests failed, say
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
Signed-off-by: Khor Shu Heng khor.heng@gojek.com
What this PR does / why we need it:
This PR add end to end test for historical feature retrieval. The new test has been tested locally against a local spark cluster, but at the moment it will fail during the CI as the corresponding test infrastructure has not been set up yet.
Which issue(s) this PR fixes:
Fixes #
Does this PR introduce a user-facing change?: