Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

fix: throw away excess data in order to avoid delivering duplicate data #1453

Merged
merged 8 commits into from
Jul 22, 2024
Merged

fix: throw away excess data in order to avoid delivering duplicate data #1453

merged 8 commits into from
Jul 22, 2024

Conversation

danieljbruce
Copy link
Contributor

@danieljbruce danieljbruce commented Jul 19, 2024 •
edited
Loading

Changes:

  • Throw away excess data if its value doesn't exceed the row key
  • Fix the mock in the Table class so that the row function actually returns a more realistic value

@product-auto-label product-auto-label bot added size: s Pull request size is small. api: bigtable Issues related to the googleapis/nodejs-bigtable API. labels Jul 19, 2024
@danieljbruce danieljbruce changed the title 353765540 try ignoring duplicated data 2 353765540 try ignoring data 2 Jul 19, 2024
@danieljbruce danieljbruce changed the title 353765540 try ignoring data 2 fix: throw away excess data Jul 19, 2024
@danieljbruce danieljbruce marked this pull request as ready for review July 19, 2024 14:33
@danieljbruce danieljbruce requested review from a team as code owners July 19, 2024 14:33
@danieljbruce danieljbruce added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 19, 2024
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 19, 2024
igorbernstein2
igorbernstein2 reviewed Jul 19, 2024
View reviewed changes
src/table.ts Outdated
@@ -759,7 +759,7 @@ Please use the format 'prezzy' or '${instance.name}/tables/prezzy'.`);
readableHighWaterMark: 0, // We need to disable readside buffering to allow for acceptable behavior when the end user cancels the stream early.
writableHighWaterMark: 0, // We need to disable writeside buffering because in nodejs 14 the call to _transform happens after write buffering. This creates problems for tracking the last seen row key.
transform(row, _encoding, callback) {
if (userCanceled) {
if (userCanceled || TableUtils.lessThanOrEqualTo(row.id, lastRowKey)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please split these into separate if statements and add a comment explaining why this is necessary

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@danieljbruce danieljbruce changed the title fix: throw away excess data fix: throw away excess data in order to avoid delivering duplicate data Jul 19, 2024
@danieljbruce danieljbruce added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 19, 2024
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 19, 2024
@danieljbruce danieljbruce added the owlbot:run Add this label to trigger the Owlbot post processor. label Jul 19, 2024
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Jul 19, 2024
@danieljbruce danieljbruce merged commit 069239d into googleapis:main Jul 22, 2024
15 of 19 checks passed
@release-please release-please bot mentioned this pull request Jul 22, 2024
Merged
@danieljbruce danieljbruce mentioned this pull request Jul 29, 2024
Open
# for free to join this conversation on GitHub. Already have an account? # to comment
Reviewers

@igorbernstein2 igorbernstein2 igorbernstein2 left review comments

@leahecole leahecole leahecole approved these changes

Assignees
No one assigned
Labels
api: bigtable Issues related to the googleapis/nodejs-bigtable API. size: s Pull request size is small.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@danieljbruce @leahecole @igorbernstein2 @yoshi-kokoro